Proposal for handling queries with parts, sets, and partitions
Hello, all. During our last telecon, I had what I thought was an idea that would unify the parallel query for sets stuff. At the time, the idea was vaguely formed, and I hadn't thought through all of the consequences. I may still not have, but I've at least got it to the point of being ready to drag it out into the light of day. Essentially, the issue is that we want to have much the same set of queries for mesh instances, parts, and partitions, but we haven't worked out how to handle this without exploding the number of functions. In the serial interface, we have lots of queries that take both a mesh instance and an entity set argument. In the now more-or-less deprecated SIDL paradigm, the mesh instance in an object (in the C++ sense), and we unified global and entity set calls by creating a root set that is, essentially, shorthand for "everything in the mesh instance". In parallel, the mesh instance resides at the process level in a partition / process / part heirarchy (black / red, green in the attached picture; yes, it's a very obvious illustration, but perhaps useful for reference) . We also want to be able to query partitions and parts (although for partitions we only want to query for numbers of things, not lists of things). Also, we'd like to able to query sets other than the root set (like that lovely blue set that spans a bunch of parts...). I propose that we think about both parts and mesh instances as coverings of the global mesh (they are), and that we also think about parts, mesh instances, and the partition as containing -all- data about some (topologically and/or geometrically compact subset) or all of the global mesh (as opposed to entity sets, which are deliberately more selective). Yes, I know they're all collections of entities, but stick with me while I justify why I think this semantic distinction is worthwhile. Also yes, we haven't decided yet (have we?) whether a part will officially contain everything or just the entities that were partitioned; I don't -think- that any of what I'm about to propose is adversely affected, either way. Finally, it's worth noting that, while a partition contains many parts distributed over many processes, each process is associated with a unique partition (MPI communicator or equivalent) and each part is associated both with a unique process and a unique partition. So specifying both part handle and partition handle is actually redundant. Given that backgroun, I propose that we overload our current query functions so that any place where a mesh instance is currently usable, a part handle or partition handle is also usable, and continue to use either a bona fide entity set or the (global placeholder) root set as the second argument. In this scenario, we would have, for instance: / partition handle getNumOfType| mesh instance , root set handle , type, result, err ) \ part handle entity set handle The other iMesh function that is obviously of this type is getNumOfTopo. A somewhat larger collection of functions would be able to take both mesh instances and part handles as the first argument, but not partition handles: getAllVtxCoords, getVtxCoordIndex, getEntities. My guess is that iterator functions will fall into the second category. Many of the set queries (number of sets, number of children, identities of sets and children) are a bit more ambiguous here, to my surprise: essentially, if you ask for the number of sets contained in a given set for a particular part, you presumably mean the number of contained sets that intersect that part. While I don't have a problem with that definition, I'm not sure it's an entirely straightforward one, either conceptually or in implementation. But I haven't thought about it in a great deal of detail, either... One way to finesse this would be to create sets at the part level rather than the mesh instance level; then all is easy again. Now, there are a couple of beneath-the-hood requirements for implementations here: 1. All part handles must be unique, even in the presence of multiple partitions. Pointer-type handles will easily satisfy this; integer-type handles may need to reserve some bits for partition ID and some for part ID. 2. In most (all?) implementations, finding the result for calls with (part handle, entity set handle...) will require an implementation to do some sort of intersection internally. This will prove especially challenging for iterator in the presence of mesh modification and/or migration. I recognize also that this paradigm shift potentially ambiguates the use of calls like addEntToSet for adding entities to a part; we can either overload those functions (probably renamed to addEntToCollection) or create new addEntToPart functions (I think there are only going to be four of these: add/rmv single/array). Okay, so there's my proposal. I'll now stand back and let people poke holes in the idea; that may not prove to be too difficult. Carl -- ------------------------------------------------------------------------ Dr. Carl Ollivier-Gooch, P.Eng. Voice: +1-604-822-1854 Associate Professor Fax: +1-604-822-2403 Department of Mechanical Engineering email: [email protected] University of British Columbia http://www.mech.ubc.ca/~cfog Vancouver, BC V6T 1Z4 http://tetra.mech.ubc.ca/ANSLab/ ------------------------------------------------------------------------
Finally, it's worth noting that, while a partition contains many parts distributed over many processes, each process is associated with a unique partition (MPI communicator or equivalent) and each part is associated both with a unique process and a unique partition. So specifying both part handle and partition handle is actually redundant.
Given that backgroun, I propose that we overload our current query functions so that any place where a mesh instance is currently usable, a part handle or partition handle is also usable, and continue to use either a bona fide entity set or the (global placeholder) root set as the second argument. In this scenario, we would have, for instance:
/ partition handle getNumOfType| mesh instance , root set handle , type, result, err ) \ part handle entity set handle
The other iMesh function that is obviously of this type is getNumOfTopo.
I may also fall into category of (for iMesh, serial) -- new enough to not be "trapped in the culture" -- so I will go ahead and ask some "non-sensical / counter-intuitive" questions (at least to me at this point of time). If mesh-instance and root-set-handle is one-to-one then specifying both seems redundant. Actually why we need both mesh-instance and entity-set-handle in functions like getNumOfType() and getNumOfTopo(), may be if entity-sets span more than one mesh instance... can they?
A somewhat larger collection of functions would be able to take both mesh instances and part handles as the first argument, but not partition handles: getAllVtxCoords, getVtxCoordIndex, getEntities.
My guess is that iterator functions will fall into the second category.
Functions like getAllVtxCoords() etc. can possibly take partition-handles as first argument like we may do in functions that support following (from combined doc. from Karen): - Map from the process rank to part handles. + Given a partition instance, return the number of on-process parts in the partition. + Given a partition instance, return all on-process part handles in the partition. - Onkar
Onkar Sahni wrote:
I may also fall into category of (for iMesh, serial) -- new enough to not be "trapped in the culture" -- so I will go ahead and ask some "non-sensical / counter-intuitive" questions (at least to me at this point of time). If mesh-instance and root-set-handle is one-to-one then specifying both seems redundant. Actually why we need both mesh-instance and entity-set-handle in functions like getNumOfType() and getNumOfTopo(), may be if entity-sets span more than one mesh instance... can they?
No, but a single mesh-instance may have many entity sets. Further, the root set is /defined/ to mean the mesh instance. It is a special value (possibly a constant) that is used to indicate that a query is to be done over the entire mesh instance, as opposed to some subset of the mesh instance indicated by some other entity set handle. We could eliminate the need for a root set at the cost of having twice as many functions in the interface. - jason
Carl Ollivier-Gooch wrote:
Hello, all.
During our last telecon, I had what I thought was an idea that would unify the parallel query for sets stuff. At the time, the idea was vaguely formed, and I hadn't thought through all of the consequences. I may still not have, but I've at least got it to the point of being ready to drag it out into the light of day.
Essentially, the issue is that we want to have much the same set of queries for mesh instances, parts, and partitions, but we haven't worked out how to handle this without exploding the number of functions.
In the serial interface, we have lots of queries that take both a mesh instance and an entity set argument. In the now more-or-less deprecated SIDL paradigm, the mesh instance in an object (in the C++ sense), and we unified global and entity set calls by creating a root set that is, essentially, shorthand for "everything in the mesh instance".
In parallel, the mesh instance resides at the process level in a partition / process / part heirarchy (black / red, green in the attached picture; yes, it's a very obvious illustration, but perhaps useful for reference) . We also want to be able to query partitions and parts (although for partitions we only want to query for numbers of things, not lists of things). Also, we'd like to able to query sets other than the root set (like that lovely blue set that spans a bunch of parts...).
I propose that we think about both parts and mesh instances as coverings of the global mesh (they are), and that we also think about parts, mesh instances, and the partition as containing -all- data about some (topologically and/or geometrically compact subset) or all of the global mesh (as opposed to entity sets, which are deliberately more selective). Yes, I know they're all collections of entities, but stick with me while I justify why I think this semantic distinction is worthwhile. Also yes, we haven't decided yet (have we?) whether a part will officially contain everything or just the entities that were partitioned; I don't -think- that any of what I'm about to propose is adversely affected, either way.
Finally, it's worth noting that, while a partition contains many parts distributed over many processes, each process is associated with a unique partition (MPI communicator or equivalent) and each part is associated both with a unique process and a unique partition. So specifying both part handle and partition handle is actually redundant.
Given that backgroun, I propose that we overload our current query functions so that any place where a mesh instance is currently usable, a part handle or partition handle is also usable, and continue to use either a bona fide entity set or the (global placeholder) root set as the second argument. In this scenario, we would have, for instance:
/ partition handle getNumOfType| mesh instance , root set handle , type, result, err ) \ part handle entity set handle
The other iMesh function that is obviously of this type is getNumOfTopo.
A somewhat larger collection of functions would be able to take both mesh instances and part handles as the first argument, but not partition handles: getAllVtxCoords, getVtxCoordIndex, getEntities.
My guess is that iterator functions will fall into the second category.
Many of the set queries (number of sets, number of children, identities of sets and children) are a bit more ambiguous here, to my surprise: essentially, if you ask for the number of sets contained in a given set for a particular part, you presumably mean the number of contained sets that intersect that part. While I don't have a problem with that definition, I'm not sure it's an entirely straightforward one, either conceptually or in implementation. But I haven't thought about it in a great deal of detail, either... One way to finesse this would be to create sets at the part level rather than the mesh instance level; then all is easy again.
Now, there are a couple of beneath-the-hood requirements for implementations here:
1. All part handles must be unique, even in the presence of multiple partitions. Pointer-type handles will easily satisfy this; integer-type handles may need to reserve some bits for partition ID and some for part ID.
But then how do we find the data for the partition once we have its 'id' from the part handle? Look it up in a table? How do we find the table?
2. In most (all?) implementations, finding the result for calls with (part handle, entity set handle...) will require an implementation to do some sort of intersection internally. This will prove especially challenging for iterator in the presence of mesh modification and/or migration.
I don't think your proposal is flawed in theory. However, it is rather difficult to implement in practice. Your assumption that the mesh_instance part of the interface is a unnecessary legacy from SIDL is incorrect. For id- or index-type handles, we need the mesh instance as a pointer to the group of data that the handles reference. We could just as easily remove the mesh_instance argument from the serial interface entirely. It is possible to embed some "instance id" in the handles, and look up the instance in some static table. And further, to be consistent, if we can remove the need for the mesh instance in some cases, we should do so for all of them. This brings us back around to passing part and partition handles as the entity set argument, as that is consistent with the serial interface. Or to look at it from the other direction: if all handles are unique across mesh instances, what is a mesh instance? The multiplexer is an example where this would be unworkable. In the multiplexer, the mesh instance is as pointer to a function table. It must be able to determine which function from which shared library a given handle is supposed to be passed to. While we can guarantee that handles are unique within an implementations, there is no way to guarantee that they are unique /between/ implementations. So as I said above, if one only considers the theoretical meaning of a mesh instance, your proposal would work. However, the mesh instance means something specific for implementations, and is necessary for any non-pointer handle implementation. - jason
Jason Kraftcheck wrote:
Carl Ollivier-Gooch wrote:
[snip]
In the serial interface, we have lots of queries that take both a mesh instance and an entity set argument. In the now more-or-less deprecated SIDL paradigm, the mesh instance in an object (in the C++ sense), and we unified global and entity set calls by creating a root set that is, essentially, shorthand for "everything in the mesh instance".
[snip]
1. All part handles must be unique, even in the presence of multiple partitions. Pointer-type handles will easily satisfy this; integer-type handles may need to reserve some bits for partition ID and some for part ID.
But then how do we find the data for the partition once we have its 'id' from the part handle? Look it up in a table? How do we find the table?
Yes. Remember, it's on the -implementation- side that this lookup has to occur, and the implementation can easily keep track of a table of partitions; this have size O(1).
I don't think your proposal is flawed in theory. However, it is rather difficult to implement in practice. Your assumption that the mesh_instance part of the interface is a unnecessary legacy from SIDL is incorrect. For id- or index-type handles, we need the mesh instance as a pointer to the group of data that the handles reference. We could just as easily remove the mesh_instance argument from the serial interface entirely. It is possible to embed some "instance id" in the handles, and look up the instance in some static table. And further, to be consistent, if we can remove the need for the mesh instance in some cases, we should do so for all of them. This brings us back around to passing part and partition handles as the entity set argument, as that is consistent with the serial interface. Or to look at it from the other direction: if all handles are unique across mesh instances, what is a mesh instance?
The multiplexer is an example where this would be unworkable. In the multiplexer, the mesh instance is as pointer to a function table. It must be able to determine which function from which shared library a given handle is supposed to be passed to. While we can guarantee that handles are unique within an implementations, there is no way to guarantee that they are unique /between/ implementations.
So as I said above, if one only considers the theoretical meaning of a mesh instance, your proposal would work. However, the mesh instance means something specific for implementations, and is necessary for any non-pointer handle implementation.
I'm sorry if I gave the impression that I think the mesh instance is "an unnecessary legacy", as that definitely wasn't my intent. And no, I don't think it's at all reasonable to expect that entity handles be unique across all mesh instances. With that cleared up, Jason, are there any points in the above that still apply? If so, let me know, and I'll try to respond to them. Carl -- ------------------------------------------------------------------------ Dr. Carl Ollivier-Gooch, P.Eng. Voice: +1-604-822-1854 Associate Professor Fax: +1-604-822-2403 Department of Mechanical Engineering email: [email protected] University of British Columbia http://www.mech.ubc.ca/~cfog Vancouver, BC V6T 1Z4 http://tetra.mech.ubc.ca/ANSLab/ ------------------------------------------------------------------------
Carl Ollivier-Gooch wrote:
Jason Kraftcheck wrote:
Carl Ollivier-Gooch wrote:
[snip]
In the serial interface, we have lots of queries that take both a mesh instance and an entity set argument. In the now more-or-less deprecated SIDL paradigm, the mesh instance in an object (in the C++ sense), and we unified global and entity set calls by creating a root set that is, essentially, shorthand for "everything in the mesh instance".
[snip]
1. All part handles must be unique, even in the presence of multiple partitions. Pointer-type handles will easily satisfy this; integer-type handles may need to reserve some bits for partition ID and some for part ID.
But then how do we find the data for the partition once we have its 'id' from the part handle? Look it up in a table? How do we find the table?
Yes. Remember, it's on the -implementation- side that this lookup has to occur, and the implementation can easily keep track of a table of partitions; this have size O(1).
Yes, the natural place to put said table is in the mesh instance.
I don't think your proposal is flawed in theory. However, it is rather difficult to implement in practice. Your assumption that the mesh_instance part of the interface is a unnecessary legacy from SIDL is incorrect. For id- or index-type handles, we need the mesh instance as a pointer to the group of data that the handles reference. We could just as easily remove the mesh_instance argument from the serial interface entirely. It is possible to embed some "instance id" in the handles, and look up the instance in some static table. And further, to be consistent, if we can remove the need for the mesh instance in some cases, we should do so for all of them. This brings us back around to passing part and partition handles as the entity set argument, as that is consistent with the serial interface. Or to look at it from the other direction: if all handles are unique across mesh instances, what is a mesh instance?
The multiplexer is an example where this would be unworkable. In the multiplexer, the mesh instance is as pointer to a function table. It must be able to determine which function from which shared library a given handle is supposed to be passed to. While we can guarantee that handles are unique within an implementations, there is no way to guarantee that they are unique /between/ implementations.
So as I said above, if one only considers the theoretical meaning of a mesh instance, your proposal would work. However, the mesh instance means something specific for implementations, and is necessary for any non-pointer handle implementation.
I'm sorry if I gave the impression that I think the mesh instance is "an unnecessary legacy", as that definitely wasn't my intent. And no, I don't think it's at all reasonable to expect that entity handles be unique across all mesh instances. With that cleared up, Jason, are there any points in the above that still apply? If so, let me know, and I'll try to respond to them.
I didn't read your initial comment about mesh instances as necessarily implying that it is "an unnecessary legacy". However, your proposal implies that. Whatever mechanism we employ to work around the lack of a mesh instance in some cases can be employed in all cases. We should therefore eliminate the mesh instance entirely, rather than making interface functions behave drastically differently in some cases. Further, your proposal in practice requires it. If a part handle is an ID and the mesh instance is a pointer, then short writing our own memory allocator, there's no way we can prevent the numerical value of the pointer from being the same as some part handle. If in the implementation we cannot determine if the passed value is the mesh instance or a part handle, then we cannot use a mesh instance in any practical way. If we remove the mesh instance from the serial interface, we're back to needing two set handles to perform the types of queries you're proposing. So why force us to jump through hoops to work without mesh instance? Just add another set handle to the functions if it is necessary. - jason
As usual, I picked the wrong telecon to miss. There should be some rule about not allowing major interface change proposals coming out on Thursday after COB (in the midwest) and essentially ratifying those changes Monday morning. But anyway... Carl Ollivier-Gooch wrote:
Hello, all.
During our last telecon, I had what I thought was an idea that would unify the parallel query for sets stuff. At the time, the idea was vaguely formed, and I hadn't thought through all of the consequences. I may still not have, but I've at least got it to the point of being ready to drag it out into the light of day.
Essentially, the issue is that we want to have much the same set of queries for mesh instances, parts, and partitions, but we haven't worked out how to handle this without exploding the number of functions.
In the serial interface, we have lots of queries that take both a mesh instance and an entity set argument. In the now more-or-less deprecated SIDL paradigm, the mesh instance in an object (in the C++ sense), and we unified global and entity set calls by creating a root set that is, essentially, shorthand for "everything in the mesh instance".
I think the above is a very useful thing to keep in mind. The two things that change when you go from there to parallel is that a) mesh instances together form a collection that in some way describes a global mesh, and b) some sets can be thought of as spanning processors, but only implicitly (that is, we don't assume the handles are the same).
In parallel, the mesh instance resides at the process level in a partition / process / part heirarchy (black / red, green in the attached picture; yes, it's a very obvious illustration, but perhaps useful for reference) . We also want to be able to query partitions and parts (although for partitions we only want to query for numbers of things, not lists of things). Also, we'd like to able to query sets other than the root set (like that lovely blue set that spans a bunch of parts...).
I think it's the other way around: the part/partition resides in the mesh instance. In parallel, there's an implicit relation between a partition on one processor and on another processor, such that both/all processors understand that their local partition information is a piece of a collection over all processors. This model reduces to the trivial case for a single processor, both for partition(s) in mesh instances and parts in partitions.
I propose that we think about both parts and mesh instances as coverings of the global mesh (they are), and that we also think about parts, mesh instances, and the partition as containing -all- data about some (topologically and/or geometrically compact subset) or all of the global mesh (as opposed to entity sets, which are deliberately more selective). Yes, I know they're all collections of entities, but stick with me while I justify why I think this semantic distinction is worthwhile. Also yes, we haven't decided yet (have we?) whether a part will officially contain everything or just the entities that were partitioned; I don't -think- that any of what I'm about to propose is adversely affected, either way.
I don't think the partition can be a covering of the whole mesh on a given processor. Given a collection of 2d and 3d elements, you need to distinguish between the 2d elements that are just adjacent to objects in the partition and the 2d elements which are in fact objects in the partition. I did not see anything following which justifies the semantic distinction you're talking about.
Finally, it's worth noting that, while a partition contains many parts distributed over many processes, each process is associated with a unique partition (MPI communicator or equivalent) and each part is associated both with a unique process and a unique partition. So specifying both part handle and partition handle is actually redundant.
Each part is associated with a unique partition and process, but each process is not associated with a unique partition. There may be multiple partitions in use at any given time, and a given process may be responsible for one or more parts in each partition. Take parallel contact detection for instance. One partition is over volume elements, the other is over faces on the skin of those elements. Processors participate in calculations for both partitions at different times. Specifying both a part and partition handle may not be strictly redundant, and in terms of implementation I don't think it's useful to to require that the partition be derivable from the part handle. I do think that in most cases, an application will know the partition that's being dealt with. For the few cases where a part is a member of multiple partitions (if we want to allow that, and I don't see a major reason why not to), there should be a function to get the partition(s) that a part is a member of.
Given that backgroun, I propose that we overload our current query functions so that any place where a mesh instance is currently usable, a part handle or partition handle is also usable, and continue to use either a bona fide entity set or the (global placeholder) root set as the second argument. In this scenario, we would have, for instance:
/ partition handle getNumOfType| mesh instance , root set handle , type, result, err ) \ part handle entity set handle
The other iMesh function that is obviously of this type is getNumOfTopo.
A somewhat larger collection of functions would be able to take both mesh instances and part handles as the first argument, but not partition handles: getAllVtxCoords, getVtxCoordIndex, getEntities.
My guess is that iterator functions will fall into the second category.
Many of the set queries (number of sets, number of children, identities of sets and children) are a bit more ambiguous here, to my surprise: essentially, if you ask for the number of sets contained in a given set for a particular part, you presumably mean the number of contained sets that intersect that part. While I don't have a problem with that definition, I'm not sure it's an entirely straightforward one, either conceptually or in implementation. But I haven't thought about it in a great deal of detail, either... One way to finesse this would be to create sets at the part level rather than the mesh instance level; then all is easy again.
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind. I know some haven't fully implemented sets, but is that going to be more difficult than the degree of interface changes being discussed here? The things that are missing in sets for use as partitions and parts are exactly the extensions we'll already need in sets for going to parallel, no matter what we choose for partition and part representation. Specifically, we'll need some notion of a correspondence between sets on different processors, to handle boundary conditions on mesh spanning processors. I'll make a diagram equivalent to Carl's today, maybe that will make things a bit clearer. - tim
Now, there are a couple of beneath-the-hood requirements for implementations here:
1. All part handles must be unique, even in the presence of multiple partitions. Pointer-type handles will easily satisfy this; integer-type handles may need to reserve some bits for partition ID and some for part ID.
2. In most (all?) implementations, finding the result for calls with (part handle, entity set handle...) will require an implementation to do some sort of intersection internally. This will prove especially challenging for iterator in the presence of mesh modification and/or migration.
I recognize also that this paradigm shift potentially ambiguates the use of calls like addEntToSet for adding entities to a part; we can either overload those functions (probably renamed to addEntToCollection) or create new addEntToPart functions (I think there are only going to be four of these: add/rmv single/array).
Okay, so there's my proposal. I'll now stand back and let people poke holes in the idea; that may not prove to be too difficult.
Carl
------------------------------------------------------------------------
-- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Argonne National Laboratory ([email protected]) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706
Hi, Tim. Actually, the situation is worse than you assumed. We did not ratify the proposal made after COB Thursday; we ratified a proposal made 40 minutes into the conference call. :) Of course, we ratified it contingent on getting feedback from you. That is why I sent the notes from yesterday's meeting. Did you get them? I'm sorry I wasn't clear in expressing that intent. We agreed that iMesh functions taking a mesh instance and an entity set handle as an argument would now also take a partition instance and part handle. These extra arguments could be NULL, providing the current iMesh serial capability. Or they could be valid handles for partitions and parts, in which case the implementation would do the appropriate set operations to return values in the entity set that are also in the part. Jason did a good job of explaining the drawbacks of the COB Thursday proposal. We also picked a date/time for the next phone conf: Jan 7 at 1:30 PST. Does that work for you? If not, we will reschedule. We want to have a parallel interface that is presentable for the next bootcamp (Mar 11) so your attendance at the phone confs is important. Karen On 12/18/07 9:39 AM, "Tim Tautges" <[email protected]> wrote:
As usual, I picked the wrong telecon to miss. There should be some rule about not allowing major interface change proposals coming out on Thursday after COB (in the midwest) and essentially ratifying those changes Monday morning. But anyway...
Carl Ollivier-Gooch wrote:
Hello, all.
During our last telecon, I had what I thought was an idea that would unify the parallel query for sets stuff. At the time, the idea was vaguely formed, and I hadn't thought through all of the consequences. I may still not have, but I've at least got it to the point of being ready to drag it out into the light of day.
Essentially, the issue is that we want to have much the same set of queries for mesh instances, parts, and partitions, but we haven't worked out how to handle this without exploding the number of functions.
In the serial interface, we have lots of queries that take both a mesh instance and an entity set argument. In the now more-or-less deprecated SIDL paradigm, the mesh instance in an object (in the C++ sense), and we unified global and entity set calls by creating a root set that is, essentially, shorthand for "everything in the mesh instance".
I think the above is a very useful thing to keep in mind. The two things that change when you go from there to parallel is that a) mesh instances together form a collection that in some way describes a global mesh, and b) some sets can be thought of as spanning processors, but only implicitly (that is, we don't assume the handles are the same).
In parallel, the mesh instance resides at the process level in a partition / process / part heirarchy (black / red, green in the attached picture; yes, it's a very obvious illustration, but perhaps useful for reference) . We also want to be able to query partitions and parts (although for partitions we only want to query for numbers of things, not lists of things). Also, we'd like to able to query sets other than the root set (like that lovely blue set that spans a bunch of parts...).
I think it's the other way around: the part/partition resides in the mesh instance. In parallel, there's an implicit relation between a partition on one processor and on another processor, such that both/all processors understand that their local partition information is a piece of a collection over all processors. This model reduces to the trivial case for a single processor, both for partition(s) in mesh instances and parts in partitions.
I propose that we think about both parts and mesh instances as coverings of the global mesh (they are), and that we also think about parts, mesh instances, and the partition as containing -all- data about some (topologically and/or geometrically compact subset) or all of the global mesh (as opposed to entity sets, which are deliberately more selective). Yes, I know they're all collections of entities, but stick with me while I justify why I think this semantic distinction is worthwhile. Also yes, we haven't decided yet (have we?) whether a part will officially contain everything or just the entities that were partitioned; I don't -think- that any of what I'm about to propose is adversely affected, either way.
I don't think the partition can be a covering of the whole mesh on a given processor. Given a collection of 2d and 3d elements, you need to distinguish between the 2d elements that are just adjacent to objects in the partition and the 2d elements which are in fact objects in the partition.
I did not see anything following which justifies the semantic distinction you're talking about.
Finally, it's worth noting that, while a partition contains many parts distributed over many processes, each process is associated with a unique partition (MPI communicator or equivalent) and each part is associated both with a unique process and a unique partition. So specifying both part handle and partition handle is actually redundant.
Each part is associated with a unique partition and process, but each process is not associated with a unique partition. There may be multiple partitions in use at any given time, and a given process may be responsible for one or more parts in each partition. Take parallel contact detection for instance. One partition is over volume elements, the other is over faces on the skin of those elements. Processors participate in calculations for both partitions at different times.
Specifying both a part and partition handle may not be strictly redundant, and in terms of implementation I don't think it's useful to to require that the partition be derivable from the part handle. I do think that in most cases, an application will know the partition that's being dealt with. For the few cases where a part is a member of multiple partitions (if we want to allow that, and I don't see a major reason why not to), there should be a function to get the partition(s) that a part is a member of.
Given that backgroun, I propose that we overload our current query functions so that any place where a mesh instance is currently usable, a part handle or partition handle is also usable, and continue to use either a bona fide entity set or the (global placeholder) root set as the second argument. In this scenario, we would have, for instance:
/ partition handle getNumOfType| mesh instance , root set handle , type, result, err ) \ part handle entity set handle
The other iMesh function that is obviously of this type is getNumOfTopo.
A somewhat larger collection of functions would be able to take both mesh instances and part handles as the first argument, but not partition handles: getAllVtxCoords, getVtxCoordIndex, getEntities.
My guess is that iterator functions will fall into the second category.
Many of the set queries (number of sets, number of children, identities of sets and children) are a bit more ambiguous here, to my surprise: essentially, if you ask for the number of sets contained in a given set for a particular part, you presumably mean the number of contained sets that intersect that part. While I don't have a problem with that definition, I'm not sure it's an entirely straightforward one, either conceptually or in implementation. But I haven't thought about it in a great deal of detail, either... One way to finesse this would be to create sets at the part level rather than the mesh instance level; then all is easy again.
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind.
I know some haven't fully implemented sets, but is that going to be more difficult than the degree of interface changes being discussed here?
The things that are missing in sets for use as partitions and parts are exactly the extensions we'll already need in sets for going to parallel, no matter what we choose for partition and part representation. Specifically, we'll need some notion of a correspondence between sets on different processors, to handle boundary conditions on mesh spanning processors.
I'll make a diagram equivalent to Carl's today, maybe that will make things a bit clearer.
- tim
Now, there are a couple of beneath-the-hood requirements for implementations here:
1. All part handles must be unique, even in the presence of multiple partitions. Pointer-type handles will easily satisfy this; integer-type handles may need to reserve some bits for partition ID and some for part ID.
2. In most (all?) implementations, finding the result for calls with (part handle, entity set handle...) will require an implementation to do some sort of intersection internally. This will prove especially challenging for iterator in the presence of mesh modification and/or migration.
I recognize also that this paradigm shift potentially ambiguates the use of calls like addEntToSet for adding entities to a part; we can either overload those functions (probably renamed to addEntToCollection) or create new addEntToPart functions (I think there are only going to be four of these: add/rmv single/array).
Okay, so there's my proposal. I'll now stand back and let people poke holes in the idea; that may not prove to be too difficult.
Carl
------------------------------------------------------------------------
Devine, Karen D. wrote:
Hi, Tim. Actually, the situation is worse than you assumed. We did not ratify the proposal made after COB Thursday; we ratified a proposal made 40 minutes into the conference call. :)
Of course, we ratified it contingent on getting feedback from you. That is why I sent the notes from yesterday's meeting. Did you get them? I'm sorry I wasn't clear in expressing that intent.
I did get them, thanks.
We agreed that iMesh functions taking a mesh instance and an entity set handle as an argument would now also take a partition instance and part handle. These extra arguments could be NULL, providing the current iMesh serial capability. Or they could be valid handles for partitions and parts, in which case the implementation would do the appropriate set operations to return values in the entity set that are also in the part. Jason did a good job of explaining the drawbacks of the COB Thursday proposal.
Yes, I just don't think it's worth all the trouble to modify the interface to that degree. I mean, why hasn't the issue of booleans on query results come up before? And if it is in fact an issue, why only on part/partition contents, and not on set query results in general? If it's only the booleans on results problem, it would be far easier (and more useful) to add the functions that work on any kind of set.
We also picked a date/time for the next phone conf: Jan 7 at 1:30 PST. Does that work for you? If not, we will reschedule. We want to have a parallel interface that is presentable for the next bootcamp (Mar 11) so your attendance at the phone confs is important.
Thanks, that works for me. My absence yesterday was a stupid calendar problem (but now I can cross off the itaps phone call app't I had marked down for this Thursday, sigh). - tim
Karen
On 12/18/07 9:39 AM, "Tim Tautges" <[email protected]> wrote:
As usual, I picked the wrong telecon to miss. There should be some rule about not allowing major interface change proposals coming out on Thursday after COB (in the midwest) and essentially ratifying those changes Monday morning. But anyway...
Carl Ollivier-Gooch wrote:
Hello, all.
During our last telecon, I had what I thought was an idea that would unify the parallel query for sets stuff. At the time, the idea was vaguely formed, and I hadn't thought through all of the consequences. I may still not have, but I've at least got it to the point of being ready to drag it out into the light of day.
Essentially, the issue is that we want to have much the same set of queries for mesh instances, parts, and partitions, but we haven't worked out how to handle this without exploding the number of functions.
In the serial interface, we have lots of queries that take both a mesh instance and an entity set argument. In the now more-or-less deprecated SIDL paradigm, the mesh instance in an object (in the C++ sense), and we unified global and entity set calls by creating a root set that is, essentially, shorthand for "everything in the mesh instance". I think the above is a very useful thing to keep in mind. The two things that change when you go from there to parallel is that a) mesh instances together form a collection that in some way describes a global mesh, and b) some sets can be thought of as spanning processors, but only implicitly (that is, we don't assume the handles are the same).
In parallel, the mesh instance resides at the process level in a partition / process / part heirarchy (black / red, green in the attached picture; yes, it's a very obvious illustration, but perhaps useful for reference) . We also want to be able to query partitions and parts (although for partitions we only want to query for numbers of things, not lists of things). Also, we'd like to able to query sets other than the root set (like that lovely blue set that spans a bunch of parts...).
I think it's the other way around: the part/partition resides in the mesh instance. In parallel, there's an implicit relation between a partition on one processor and on another processor, such that both/all processors understand that their local partition information is a piece of a collection over all processors. This model reduces to the trivial case for a single processor, both for partition(s) in mesh instances and parts in partitions.
I propose that we think about both parts and mesh instances as coverings of the global mesh (they are), and that we also think about parts, mesh instances, and the partition as containing -all- data about some (topologically and/or geometrically compact subset) or all of the global mesh (as opposed to entity sets, which are deliberately more selective). Yes, I know they're all collections of entities, but stick with me while I justify why I think this semantic distinction is worthwhile. Also yes, we haven't decided yet (have we?) whether a part will officially contain everything or just the entities that were partitioned; I don't -think- that any of what I'm about to propose is adversely affected, either way.
I don't think the partition can be a covering of the whole mesh on a given processor. Given a collection of 2d and 3d elements, you need to distinguish between the 2d elements that are just adjacent to objects in the partition and the 2d elements which are in fact objects in the partition.
I did not see anything following which justifies the semantic distinction you're talking about.
Finally, it's worth noting that, while a partition contains many parts distributed over many processes, each process is associated with a unique partition (MPI communicator or equivalent) and each part is associated both with a unique process and a unique partition. So specifying both part handle and partition handle is actually redundant.
Each part is associated with a unique partition and process, but each process is not associated with a unique partition. There may be multiple partitions in use at any given time, and a given process may be responsible for one or more parts in each partition. Take parallel contact detection for instance. One partition is over volume elements, the other is over faces on the skin of those elements. Processors participate in calculations for both partitions at different times.
Specifying both a part and partition handle may not be strictly redundant, and in terms of implementation I don't think it's useful to to require that the partition be derivable from the part handle. I do think that in most cases, an application will know the partition that's being dealt with. For the few cases where a part is a member of multiple partitions (if we want to allow that, and I don't see a major reason why not to), there should be a function to get the partition(s) that a part is a member of.
Given that backgroun, I propose that we overload our current query functions so that any place where a mesh instance is currently usable, a part handle or partition handle is also usable, and continue to use either a bona fide entity set or the (global placeholder) root set as the second argument. In this scenario, we would have, for instance:
/ partition handle getNumOfType| mesh instance , root set handle , type, result, err ) \ part handle entity set handle
The other iMesh function that is obviously of this type is getNumOfTopo.
A somewhat larger collection of functions would be able to take both mesh instances and part handles as the first argument, but not partition handles: getAllVtxCoords, getVtxCoordIndex, getEntities.
My guess is that iterator functions will fall into the second category.
Many of the set queries (number of sets, number of children, identities of sets and children) are a bit more ambiguous here, to my surprise: essentially, if you ask for the number of sets contained in a given set for a particular part, you presumably mean the number of contained sets that intersect that part. While I don't have a problem with that definition, I'm not sure it's an entirely straightforward one, either conceptually or in implementation. But I haven't thought about it in a great deal of detail, either... One way to finesse this would be to create sets at the part level rather than the mesh instance level; then all is easy again.
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind.
I know some haven't fully implemented sets, but is that going to be more difficult than the degree of interface changes being discussed here?
The things that are missing in sets for use as partitions and parts are exactly the extensions we'll already need in sets for going to parallel, no matter what we choose for partition and part representation. Specifically, we'll need some notion of a correspondence between sets on different processors, to handle boundary conditions on mesh spanning processors.
I'll make a diagram equivalent to Carl's today, maybe that will make things a bit clearer.
- tim
Now, there are a couple of beneath-the-hood requirements for implementations here:
1. All part handles must be unique, even in the presence of multiple partitions. Pointer-type handles will easily satisfy this; integer-type handles may need to reserve some bits for partition ID and some for part ID.
2. In most (all?) implementations, finding the result for calls with (part handle, entity set handle...) will require an implementation to do some sort of intersection internally. This will prove especially challenging for iterator in the presence of mesh modification and/or migration.
I recognize also that this paradigm shift potentially ambiguates the use of calls like addEntToSet for adding entities to a part; we can either overload those functions (probably renamed to addEntToCollection) or create new addEntToPart functions (I think there are only going to be four of these: add/rmv single/array).
Okay, so there's my proposal. I'll now stand back and let people poke holes in the idea; that may not prove to be too difficult.
Carl
------------------------------------------------------------------------
-- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Argonne National Laboratory ([email protected]) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706
On Tue, 2007-12-18 at 12:17 -0600, Tim Tautges wrote:
Devine, Karen D. wrote:
Hi, Tim. Actually, the situation is worse than you assumed. We did not ratify the proposal made after COB Thursday; we ratified a proposal made 40 minutes into the conference call. :)
Of course, we ratified it contingent on getting feedback from you. That is why I sent the notes from yesterday's meeting. Did you get them? I'm sorry I wasn't clear in expressing that intent.
I did get them, thanks.
We agreed that iMesh functions taking a mesh instance and an entity set handle as an argument would now also take a partition instance and part handle. These extra arguments could be NULL, providing the current iMesh serial capability. Or they could be valid handles for partitions and parts, in which case the implementation would do the appropriate set operations to return values in the entity set that are also in the part. Jason did a good job of explaining the drawbacks of the COB Thursday proposal.
Yes, I just don't think it's worth all the trouble to modify the interface to that degree. I mean, why hasn't the issue of booleans on query results come up before? And if it is in fact an issue, why only on part/partition contents, and not on set query results in general? If it's only the booleans on results problem, it would be far easier (and more useful) to add the functions that work on any kind of set.
If meshes and partitions are parallel and not hierarchically related, these boolean function would avoid the application having to query for sets out of each hierarchy and intersecting the resulting sets.
We also picked a date/time for the next phone conf: Jan 7 at 1:30 PST. Does that work for you? If not, we will reschedule. We want to have a parallel interface that is presentable for the next bootcamp (Mar 11) so your attendance at the phone confs is important.
Thanks, that works for me. My absence yesterday was a stupid calendar problem (but now I can cross off the itaps phone call app't I had marked down for this Thursday, sigh).
- tim
Karen
On 12/18/07 9:39 AM, "Tim Tautges" <[email protected]> wrote:
As usual, I picked the wrong telecon to miss. There should be some rule about not allowing major interface change proposals coming out on Thursday after COB (in the midwest) and essentially ratifying those changes Monday morning. But anyway...
Carl Ollivier-Gooch wrote:
Hello, all.
During our last telecon, I had what I thought was an idea that would unify the parallel query for sets stuff. At the time, the idea was vaguely formed, and I hadn't thought through all of the consequences. I may still not have, but I've at least got it to the point of being ready to drag it out into the light of day.
Essentially, the issue is that we want to have much the same set of queries for mesh instances, parts, and partitions, but we haven't worked out how to handle this without exploding the number of functions.
In the serial interface, we have lots of queries that take both a mesh instance and an entity set argument. In the now more-or-less deprecated SIDL paradigm, the mesh instance in an object (in the C++ sense), and we unified global and entity set calls by creating a root set that is, essentially, shorthand for "everything in the mesh instance". I think the above is a very useful thing to keep in mind. The two things that change when you go from there to parallel is that a) mesh instances together form a collection that in some way describes a global mesh, and b) some sets can be thought of as spanning processors, but only implicitly (that is, we don't assume the handles are the same).
In parallel, the mesh instance resides at the process level in a partition / process / part heirarchy (black / red, green in the attached picture; yes, it's a very obvious illustration, but perhaps useful for reference) . We also want to be able to query partitions and parts (although for partitions we only want to query for numbers of things, not lists of things). Also, we'd like to able to query sets other than the root set (like that lovely blue set that spans a bunch of parts...).
I think it's the other way around: the part/partition resides in the mesh instance. In parallel, there's an implicit relation between a partition on one processor and on another processor, such that both/all processors understand that their local partition information is a piece of a collection over all processors. This model reduces to the trivial case for a single processor, both for partition(s) in mesh instances and parts in partitions.
I propose that we think about both parts and mesh instances as coverings of the global mesh (they are), and that we also think about parts, mesh instances, and the partition as containing -all- data about some (topologically and/or geometrically compact subset) or all of the global mesh (as opposed to entity sets, which are deliberately more selective). Yes, I know they're all collections of entities, but stick with me while I justify why I think this semantic distinction is worthwhile. Also yes, we haven't decided yet (have we?) whether a part will officially contain everything or just the entities that were partitioned; I don't -think- that any of what I'm about to propose is adversely affected, either way.
I don't think the partition can be a covering of the whole mesh on a given processor. Given a collection of 2d and 3d elements, you need to distinguish between the 2d elements that are just adjacent to objects in the partition and the 2d elements which are in fact objects in the partition.
I did not see anything following which justifies the semantic distinction you're talking about.
Finally, it's worth noting that, while a partition contains many parts distributed over many processes, each process is associated with a unique partition (MPI communicator or equivalent) and each part is associated both with a unique process and a unique partition. So specifying both part handle and partition handle is actually redundant.
Each part is associated with a unique partition and process, but each process is not associated with a unique partition. There may be multiple partitions in use at any given time, and a given process may be responsible for one or more parts in each partition. Take parallel contact detection for instance. One partition is over volume elements, the other is over faces on the skin of those elements. Processors participate in calculations for both partitions at different times.
Specifying both a part and partition handle may not be strictly redundant, and in terms of implementation I don't think it's useful to to require that the partition be derivable from the part handle. I do think that in most cases, an application will know the partition that's being dealt with. For the few cases where a part is a member of multiple partitions (if we want to allow that, and I don't see a major reason why not to), there should be a function to get the partition(s) that a part is a member of.
Given that backgroun, I propose that we overload our current query functions so that any place where a mesh instance is currently usable, a part handle or partition handle is also usable, and continue to use either a bona fide entity set or the (global placeholder) root set as the second argument. In this scenario, we would have, for instance:
/ partition handle getNumOfType| mesh instance , root set handle , type, result, err ) \ part handle entity set handle
The other iMesh function that is obviously of this type is getNumOfTopo.
A somewhat larger collection of functions would be able to take both mesh instances and part handles as the first argument, but not partition handles: getAllVtxCoords, getVtxCoordIndex, getEntities.
My guess is that iterator functions will fall into the second category.
Many of the set queries (number of sets, number of children, identities of sets and children) are a bit more ambiguous here, to my surprise: essentially, if you ask for the number of sets contained in a given set for a particular part, you presumably mean the number of contained sets that intersect that part. While I don't have a problem with that definition, I'm not sure it's an entirely straightforward one, either conceptually or in implementation. But I haven't thought about it in a great deal of detail, either... One way to finesse this would be to create sets at the part level rather than the mesh instance level; then all is easy again.
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind.
I know some haven't fully implemented sets, but is that going to be more difficult than the degree of interface changes being discussed here?
The things that are missing in sets for use as partitions and parts are exactly the extensions we'll already need in sets for going to parallel, no matter what we choose for partition and part representation. Specifically, we'll need some notion of a correspondence between sets on different processors, to handle boundary conditions on mesh spanning processors.
I'll make a diagram equivalent to Carl's today, maybe that will make things a bit clearer.
- tim
Now, there are a couple of beneath-the-hood requirements for implementations here:
1. All part handles must be unique, even in the presence of multiple partitions. Pointer-type handles will easily satisfy this; integer-type handles may need to reserve some bits for partition ID and some for part ID.
2. In most (all?) implementations, finding the result for calls with (part handle, entity set handle...) will require an implementation to do some sort of intersection internally. This will prove especially challenging for iterator in the presence of mesh modification and/or migration.
I recognize also that this paradigm shift potentially ambiguates the use of calls like addEntToSet for adding entities to a part; we can either overload those functions (probably renamed to addEntToCollection) or create new addEntToPart functions (I think there are only going to be four of these: add/rmv single/array).
Okay, so there's my proposal. I'll now stand back and let people poke holes in the idea; that may not prove to be too difficult.
Carl
------------------------------------------------------------------------
Vitus Leung wrote:
Yes, I just don't think it's worth all the trouble to modify the interface to that degree. I mean, why hasn't the issue of booleans on query results come up before? And if it is in fact an issue, why only on part/partition contents, and not on set query results in general? If it's only the booleans on results problem, it would be far easier (and more useful) to add the functions that work on any kind of set.
If meshes and partitions are parallel and not hierarchically related, these boolean function would avoid the application having to query for sets out of each hierarchy and intersecting the resulting sets.
What do you mean by a mesh? The set of equal-dimensional entities read from a file? The contents of a given file? Surface elements from a membrane physics module interfacing with volume elements from a cfd module? It's quite easy to come up with a situation where just about any plausible grouping of entities can be viewed as some arbitrary set of entities. In serial, the parts in a partition don't have any special significance, other than they have a covering property over another (possibly arbitrary) group of entities. So, calling something a "mesh" doesn't carry the special significance from the point of view of the iMesh implementation, even though it may be quite significant to the application. If the cost of booleans in the application is of such great concern, the results of these queries can be put in their own entity sets, which can be input to existing boolean functions in the interface. In some cases that's desirable, while in other cases the application may not require storing the results down in the implementation. - tim
We also picked a date/time for the next phone conf: Jan 7 at 1:30 PST. Does that work for you? If not, we will reschedule. We want to have a parallel interface that is presentable for the next bootcamp (Mar 11) so your attendance at the phone confs is important.
Thanks, that works for me. My absence yesterday was a stupid calendar problem (but now I can cross off the itaps phone call app't I had marked down for this Thursday, sigh).
- tim
Karen
On 12/18/07 9:39 AM, "Tim Tautges" <[email protected]> wrote:
As usual, I picked the wrong telecon to miss. There should be some rule about not allowing major interface change proposals coming out on Thursday after COB (in the midwest) and essentially ratifying those changes Monday morning. But anyway...
Carl Ollivier-Gooch wrote:
Hello, all.
During our last telecon, I had what I thought was an idea that would unify the parallel query for sets stuff. At the time, the idea was vaguely formed, and I hadn't thought through all of the consequences. I may still not have, but I've at least got it to the point of being ready to drag it out into the light of day.
Essentially, the issue is that we want to have much the same set of queries for mesh instances, parts, and partitions, but we haven't worked out how to handle this without exploding the number of functions.
In the serial interface, we have lots of queries that take both a mesh instance and an entity set argument. In the now more-or-less deprecated SIDL paradigm, the mesh instance in an object (in the C++ sense), and we unified global and entity set calls by creating a root set that is, essentially, shorthand for "everything in the mesh instance". I think the above is a very useful thing to keep in mind. The two things that change when you go from there to parallel is that a) mesh instances together form a collection that in some way describes a global mesh, and b) some sets can be thought of as spanning processors, but only implicitly (that is, we don't assume the handles are the same).
In parallel, the mesh instance resides at the process level in a partition / process / part heirarchy (black / red, green in the attached picture; yes, it's a very obvious illustration, but perhaps useful for reference) . We also want to be able to query partitions and parts (although for partitions we only want to query for numbers of things, not lists of things). Also, we'd like to able to query sets other than the root set (like that lovely blue set that spans a bunch of parts...).
I think it's the other way around: the part/partition resides in the mesh instance. In parallel, there's an implicit relation between a partition on one processor and on another processor, such that both/all processors understand that their local partition information is a piece of a collection over all processors. This model reduces to the trivial case for a single processor, both for partition(s) in mesh instances and parts in partitions.
I propose that we think about both parts and mesh instances as coverings of the global mesh (they are), and that we also think about parts, mesh instances, and the partition as containing -all- data about some (topologically and/or geometrically compact subset) or all of the global mesh (as opposed to entity sets, which are deliberately more selective). Yes, I know they're all collections of entities, but stick with me while I justify why I think this semantic distinction is worthwhile. Also yes, we haven't decided yet (have we?) whether a part will officially contain everything or just the entities that were partitioned; I don't -think- that any of what I'm about to propose is adversely affected, either way.
I don't think the partition can be a covering of the whole mesh on a given processor. Given a collection of 2d and 3d elements, you need to distinguish between the 2d elements that are just adjacent to objects in the partition and the 2d elements which are in fact objects in the partition.
I did not see anything following which justifies the semantic distinction you're talking about.
Finally, it's worth noting that, while a partition contains many parts distributed over many processes, each process is associated with a unique partition (MPI communicator or equivalent) and each part is associated both with a unique process and a unique partition. So specifying both part handle and partition handle is actually redundant.
Each part is associated with a unique partition and process, but each process is not associated with a unique partition. There may be multiple partitions in use at any given time, and a given process may be responsible for one or more parts in each partition. Take parallel contact detection for instance. One partition is over volume elements, the other is over faces on the skin of those elements. Processors participate in calculations for both partitions at different times.
Specifying both a part and partition handle may not be strictly redundant, and in terms of implementation I don't think it's useful to to require that the partition be derivable from the part handle. I do think that in most cases, an application will know the partition that's being dealt with. For the few cases where a part is a member of multiple partitions (if we want to allow that, and I don't see a major reason why not to), there should be a function to get the partition(s) that a part is a member of.
Given that backgroun, I propose that we overload our current query functions so that any place where a mesh instance is currently usable, a part handle or partition handle is also usable, and continue to use either a bona fide entity set or the (global placeholder) root set as the second argument. In this scenario, we would have, for instance:
/ partition handle getNumOfType| mesh instance , root set handle , type, result, err ) \ part handle entity set handle
The other iMesh function that is obviously of this type is getNumOfTopo.
A somewhat larger collection of functions would be able to take both mesh instances and part handles as the first argument, but not partition handles: getAllVtxCoords, getVtxCoordIndex, getEntities.
My guess is that iterator functions will fall into the second category.
Many of the set queries (number of sets, number of children, identities of sets and children) are a bit more ambiguous here, to my surprise: essentially, if you ask for the number of sets contained in a given set for a particular part, you presumably mean the number of contained sets that intersect that part. While I don't have a problem with that definition, I'm not sure it's an entirely straightforward one, either conceptually or in implementation. But I haven't thought about it in a great deal of detail, either... One way to finesse this would be to create sets at the part level rather than the mesh instance level; then all is easy again.
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind.
I know some haven't fully implemented sets, but is that going to be more difficult than the degree of interface changes being discussed here?
The things that are missing in sets for use as partitions and parts are exactly the extensions we'll already need in sets for going to parallel, no matter what we choose for partition and part representation. Specifically, we'll need some notion of a correspondence between sets on different processors, to handle boundary conditions on mesh spanning processors.
I'll make a diagram equivalent to Carl's today, maybe that will make things a bit clearer.
- tim
Now, there are a couple of beneath-the-hood requirements for implementations here:
1. All part handles must be unique, even in the presence of multiple partitions. Pointer-type handles will easily satisfy this; integer-type handles may need to reserve some bits for partition ID and some for part ID.
2. In most (all?) implementations, finding the result for calls with (part handle, entity set handle...) will require an implementation to do some sort of intersection internally. This will prove especially challenging for iterator in the presence of mesh modification and/or migration.
I recognize also that this paradigm shift potentially ambiguates the use of calls like addEntToSet for adding entities to a part; we can either overload those functions (probably renamed to addEntToCollection) or create new addEntToPart functions (I think there are only going to be four of these: add/rmv single/array).
Okay, so there's my proposal. I'll now stand back and let people poke holes in the idea; that may not prove to be too difficult.
Carl
------------------------------------------------------------------------
-- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Argonne National Laboratory ([email protected]) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706
On Tue, 2007-12-18 at 10:39 -0600, Tim Tautges wrote:
As usual, I picked the wrong telecon to miss. There should be some rule about not allowing major interface change proposals coming out on Thursday after COB (in the midwest) and essentially ratifying those changes Monday morning. But anyway...
Carl Ollivier-Gooch wrote:
Hello, all.
During our last telecon, I had what I thought was an idea that would unify the parallel query for sets stuff. At the time, the idea was vaguely formed, and I hadn't thought through all of the consequences. I may still not have, but I've at least got it to the point of being ready to drag it out into the light of day.
Essentially, the issue is that we want to have much the same set of queries for mesh instances, parts, and partitions, but we haven't worked out how to handle this without exploding the number of functions.
In the serial interface, we have lots of queries that take both a mesh instance and an entity set argument. In the now more-or-less deprecated SIDL paradigm, the mesh instance in an object (in the C++ sense), and we unified global and entity set calls by creating a root set that is, essentially, shorthand for "everything in the mesh instance".
I think the above is a very useful thing to keep in mind. The two things that change when you go from there to parallel is that a) mesh instances together form a collection that in some way describes a global mesh, and b) some sets can be thought of as spanning processors, but only implicitly (that is, we don't assume the handles are the same).
In parallel, the mesh instance resides at the process level in a partition / process / part heirarchy (black / red, green in the attached picture; yes, it's a very obvious illustration, but perhaps useful for reference) . We also want to be able to query partitions and parts (although for partitions we only want to query for numbers of things, not lists of things). Also, we'd like to able to query sets other than the root set (like that lovely blue set that spans a bunch of parts...).
I think it's the other way around: the part/partition resides in the mesh instance. In parallel, there's an implicit relation between a partition on one processor and on another processor, such that both/all processors understand that their local partition information is a piece of a collection over all processors. This model reduces to the trivial case for a single processor, both for partition(s) in mesh instances and parts in partitions.
I thought that the mesh instance for a processor is what resides on that processor alone. This would make it impossible for a mesh instance to contain a partition.
I propose that we think about both parts and mesh instances as coverings of the global mesh (they are), and that we also think about parts, mesh instances, and the partition as containing -all- data about some (topologically and/or geometrically compact subset) or all of the global mesh (as opposed to entity sets, which are deliberately more selective). Yes, I know they're all collections of entities, but stick with me while I justify why I think this semantic distinction is worthwhile. Also yes, we haven't decided yet (have we?) whether a part will officially contain everything or just the entities that were partitioned; I don't -think- that any of what I'm about to propose is adversely affected, either way.
I don't think the partition can be a covering of the whole mesh on a given processor. Given a collection of 2d and 3d elements, you need to distinguish between the 2d elements that are just adjacent to objects in the partition and the 2d elements which are in fact objects in the partition.
I did not see anything following which justifies the semantic distinction you're talking about.
Finally, it's worth noting that, while a partition contains many parts distributed over many processes, each process is associated with a unique partition (MPI communicator or equivalent) and each part is associated both with a unique process and a unique partition. So specifying both part handle and partition handle is actually redundant.
Each part is associated with a unique partition and process, but each process is not associated with a unique partition. There may be multiple partitions in use at any given time, and a given process may be responsible for one or more parts in each partition. Take parallel contact detection for instance. One partition is over volume elements, the other is over faces on the skin of those elements. Processors participate in calculations for both partitions at different times.
Specifying both a part and partition handle may not be strictly redundant, and in terms of implementation I don't think it's useful to to require that the partition be derivable from the part handle. I do think that in most cases, an application will know the partition that's being dealt with. For the few cases where a part is a member of multiple partitions (if we want to allow that, and I don't see a major reason why not to), there should be a function to get the partition(s) that a part is a member of.
Given that backgroun, I propose that we overload our current query functions so that any place where a mesh instance is currently usable, a part handle or partition handle is also usable, and continue to use either a bona fide entity set or the (global placeholder) root set as the second argument. In this scenario, we would have, for instance:
/ partition handle getNumOfType| mesh instance , root set handle , type, result, err ) \ part handle entity set handle
The other iMesh function that is obviously of this type is getNumOfTopo.
A somewhat larger collection of functions would be able to take both mesh instances and part handles as the first argument, but not partition handles: getAllVtxCoords, getVtxCoordIndex, getEntities.
My guess is that iterator functions will fall into the second category.
Many of the set queries (number of sets, number of children, identities of sets and children) are a bit more ambiguous here, to my surprise: essentially, if you ask for the number of sets contained in a given set for a particular part, you presumably mean the number of contained sets that intersect that part. While I don't have a problem with that definition, I'm not sure it's an entirely straightforward one, either conceptually or in implementation. But I haven't thought about it in a great deal of detail, either... One way to finesse this would be to create sets at the part level rather than the mesh instance level; then all is easy again.
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind.
I know some haven't fully implemented sets, but is that going to be more difficult than the degree of interface changes being discussed here?
The things that are missing in sets for use as partitions and parts are exactly the extensions we'll already need in sets for going to parallel, no matter what we choose for partition and part representation. Specifically, we'll need some notion of a correspondence between sets on different processors, to handle boundary conditions on mesh spanning processors.
I'll make a diagram equivalent to Carl's today, maybe that will make things a bit clearer.
- tim
Now, there are a couple of beneath-the-hood requirements for implementations here:
1. All part handles must be unique, even in the presence of multiple partitions. Pointer-type handles will easily satisfy this; integer-type handles may need to reserve some bits for partition ID and some for part ID.
2. In most (all?) implementations, finding the result for calls with (part handle, entity set handle...) will require an implementation to do some sort of intersection internally. This will prove especially challenging for iterator in the presence of mesh modification and/or migration.
I recognize also that this paradigm shift potentially ambiguates the use of calls like addEntToSet for adding entities to a part; we can either overload those functions (probably renamed to addEntToCollection) or create new addEntToPart functions (I think there are only going to be four of these: add/rmv single/array).
Okay, so there's my proposal. I'll now stand back and let people poke holes in the idea; that may not prove to be too difficult.
Carl
------------------------------------------------------------------------
Vitus Leung wrote:
I think it's the other way around: the part/partition resides in the mesh instance. In parallel, there's an implicit relation between a partition on one processor and on another processor, such that both/all processors understand that their local partition information is a piece of a collection over all processors. This model reduces to the trivial case for a single processor, both for partition(s) in mesh instances and parts in partitions.
I thought that the mesh instance for a processor is what resides on that processor alone. This would make it impossible for a mesh instance to contain a partition.
The mesh instance is what resides on that processor, but there's also an implicit relation between instances and partitions when running in parallel. Think of running the same mesh in serial then on 2 procs then on 4096. You want the same boundary condition sets to exist over the whole mesh in each case, and in each case you have a partition, possibly with exactly the same number of parts in all cases. In serial the instance happens to contain the partition and all the parts; in parallel each instance contains a partition set, understood to be the same as the partition sets on the other processors, and the parts which are local/assigned to that processor. Similarly, each processor has a boundary condition set, understood to be the same as the boundary condition sets on other processors, and the entities which reside locally on that processor in that set. If a given processor doesn't have any entities locally in that set, that processor may or may not have that (empty) set. For MOAB, the application (parallel meshing, in this case) tells the parallel writer which tag[/value] combinations should be looked for to find sets spanning processors; thus, you can have an arbitrary number of sets, of multiple types, spanning or not spanning processors. I imagine a reader might keep information about that around after reading, though I haven't implemented that yet. - tim
I propose that we think about both parts and mesh instances as coverings of the global mesh (they are), and that we also think about parts, mesh instances, and the partition as containing -all- data about some (topologically and/or geometrically compact subset) or all of the global mesh (as opposed to entity sets, which are deliberately more selective). Yes, I know they're all collections of entities, but stick with me while I justify why I think this semantic distinction is worthwhile. Also yes, we haven't decided yet (have we?) whether a part will officially contain everything or just the entities that were partitioned; I don't -think- that any of what I'm about to propose is adversely affected, either way.
I don't think the partition can be a covering of the whole mesh on a given processor. Given a collection of 2d and 3d elements, you need to distinguish between the 2d elements that are just adjacent to objects in the partition and the 2d elements which are in fact objects in the partition.
I did not see anything following which justifies the semantic distinction you're talking about.
Finally, it's worth noting that, while a partition contains many parts distributed over many processes, each process is associated with a unique partition (MPI communicator or equivalent) and each part is associated both with a unique process and a unique partition. So specifying both part handle and partition handle is actually redundant.
Each part is associated with a unique partition and process, but each process is not associated with a unique partition. There may be multiple partitions in use at any given time, and a given process may be responsible for one or more parts in each partition. Take parallel contact detection for instance. One partition is over volume elements, the other is over faces on the skin of those elements. Processors participate in calculations for both partitions at different times.
Specifying both a part and partition handle may not be strictly redundant, and in terms of implementation I don't think it's useful to to require that the partition be derivable from the part handle. I do think that in most cases, an application will know the partition that's being dealt with. For the few cases where a part is a member of multiple partitions (if we want to allow that, and I don't see a major reason why not to), there should be a function to get the partition(s) that a part is a member of.
Given that backgroun, I propose that we overload our current query functions so that any place where a mesh instance is currently usable, a part handle or partition handle is also usable, and continue to use either a bona fide entity set or the (global placeholder) root set as the second argument. In this scenario, we would have, for instance:
/ partition handle getNumOfType| mesh instance , root set handle , type, result, err ) \ part handle entity set handle
The other iMesh function that is obviously of this type is getNumOfTopo.
A somewhat larger collection of functions would be able to take both mesh instances and part handles as the first argument, but not partition handles: getAllVtxCoords, getVtxCoordIndex, getEntities.
My guess is that iterator functions will fall into the second category.
Many of the set queries (number of sets, number of children, identities of sets and children) are a bit more ambiguous here, to my surprise: essentially, if you ask for the number of sets contained in a given set for a particular part, you presumably mean the number of contained sets that intersect that part. While I don't have a problem with that definition, I'm not sure it's an entirely straightforward one, either conceptually or in implementation. But I haven't thought about it in a great deal of detail, either... One way to finesse this would be to create sets at the part level rather than the mesh instance level; then all is easy again.
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind.
I know some haven't fully implemented sets, but is that going to be more difficult than the degree of interface changes being discussed here?
The things that are missing in sets for use as partitions and parts are exactly the extensions we'll already need in sets for going to parallel, no matter what we choose for partition and part representation. Specifically, we'll need some notion of a correspondence between sets on different processors, to handle boundary conditions on mesh spanning processors.
I'll make a diagram equivalent to Carl's today, maybe that will make things a bit clearer.
- tim
Now, there are a couple of beneath-the-hood requirements for implementations here:
1. All part handles must be unique, even in the presence of multiple partitions. Pointer-type handles will easily satisfy this; integer-type handles may need to reserve some bits for partition ID and some for part ID.
2. In most (all?) implementations, finding the result for calls with (part handle, entity set handle...) will require an implementation to do some sort of intersection internally. This will prove especially challenging for iterator in the presence of mesh modification and/or migration.
I recognize also that this paradigm shift potentially ambiguates the use of calls like addEntToSet for adding entities to a part; we can either overload those functions (probably renamed to addEntToCollection) or create new addEntToPart functions (I think there are only going to be four of these: add/rmv single/array).
Okay, so there's my proposal. I'll now stand back and let people poke holes in the idea; that may not prove to be too difficult.
Carl
------------------------------------------------------------------------
-- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Argonne National Laboratory ([email protected]) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind.
If "sets as both parts and partitions" mean using entity-sets then we have already discussed this in great detail. I think the question in discussion is irrespective of how one implements parts and partitions? If I understood the question right then we need to answer queries which involve a specific part and a specific entity-set (where entity-set may span more than one part, may be local to a process). Now, if it is implied that choosing entity-sets for both parts and partitions allows to use set operations (booleans), outside the interface explicitly by applications, to answer such queries well then applications might as well do it in other equivalent ways (for example, loop over entities in set and ask residence parts to get entities in the entity-sey belonging to a specific part). - Onkar
Since the first thing that was agreed to is that there would be the ability to deal parts that does not require an implementation to do things via sets (reverse classification), I sure hope we are not getting this issue confused again. As discussed in some technical detail at the boot camp, there are two high level approaches one based on classification (no use of sets) and reverse classification (what sets do well). All the SCOREC tools that are being used for parallel adaptive simulations, and (in the past) for parallel mesh generation, are based on a classification approach. Onkar Sahni wrote:
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind.
If "sets as both parts and partitions" mean using entity-sets then we have already discussed this in great detail. I think the question in discussion is irrespective of how one implements parts and partitions? If I understood the question right then we need to answer queries which involve a specific part and a specific entity-set (where entity-set may span more than one part, may be local to a process). Now, if it is implied that choosing entity-sets for both parts and partitions allows to use set operations (booleans), outside the interface explicitly by applications, to answer such queries well then applications might as well do it in other equivalent ways (for example, loop over entities in set and ask residence parts to get entities in the entity-sey belonging to a specific part).
- Onkar
Then why do we need functions operating on parts or partitions at all? Those are reverse classification-type queries, right? - tim Mark Shephard wrote:
Since the first thing that was agreed to is that there would be the ability to deal parts that does not require an implementation to do things via sets (reverse classification), I sure hope we are not getting this issue confused again. As discussed in some technical detail at the boot camp, there are two high level approaches one based on classification (no use of sets) and reverse classification (what sets do well). All the SCOREC tools that are being used for parallel adaptive simulations, and (in the past) for parallel mesh generation, are based on a classification approach.
Onkar Sahni wrote:
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind.
If "sets as both parts and partitions" mean using entity-sets then we have already discussed this in great detail. I think the question in discussion is irrespective of how one implements parts and partitions? If I understood the question right then we need to answer queries which involve a specific part and a specific entity-set (where entity-set may span more than one part, may be local to a process). Now, if it is implied that choosing entity-sets for both parts and partitions allows to use set operations (booleans), outside the interface explicitly by applications, to answer such queries well then applications might as well do it in other equivalent ways (for example, loop over entities in set and ask residence parts to get entities in the entity-sey belonging to a specific part).
- Onkar
-- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Argonne National Laboratory ([email protected]) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706
Tim, No that is not true!!! The questions asked by an interface do not require specific information be stored. It requires the ability to answer the question!!!!! We very much use a partition model without sets or explicit storage of reverse classification. Reasons for this are explained in various papers. However, knowing that is less the issue. The reason for interface API is to allow people to answer the questions, not force them to use specific structures. Mark Tim Tautges wrote:
Then why do we need functions operating on parts or partitions at all? Those are reverse classification-type queries, right?
- tim
Mark Shephard wrote:
Since the first thing that was agreed to is that there would be the ability to deal parts that does not require an implementation to do things via sets (reverse classification), I sure hope we are not getting this issue confused again. As discussed in some technical detail at the boot camp, there are two high level approaches one based on classification (no use of sets) and reverse classification (what sets do well). All the SCOREC tools that are being used for parallel adaptive simulations, and (in the past) for parallel mesh generation, are based on a classification approach.
Onkar Sahni wrote:
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind.
If "sets as both parts and partitions" mean using entity-sets then we have already discussed this in great detail. I think the question in discussion is irrespective of how one implements parts and partitions? If I understood the question right then we need to answer queries which involve a specific part and a specific entity-set (where entity-set may span more than one part, may be local to a process). Now, if it is implied that choosing entity-sets for both parts and partitions allows to use set operations (booleans), outside the interface explicitly by applications, to answer such queries well then applications might as well do it in other equivalent ways (for example, loop over entities in set and ask residence parts to get entities in the entity-sey belonging to a specific part).
- Onkar
But the language and constructs used to answer the questions (and to ask the questions) is what we call the data model. For example, the data model says we pass references to entities back and forth using handles. As proposed by you, the data model for parts refers to them using integer ids. I think those ids could just as easily be handles, and your implementation could implement those handles as integers. As for how those are actually represented underneath, I strongly agree we shouldn't force implementations to use one thing or another. I'm just saying the handles used to refer to parts and partitions should also be usable in the other functions that deal with sets. I wasn't very careful in making this distinction in my writeup I sent earlier. So, to go back to my original question, and back to what we had agreed going out of the last bootcamp: why can't we pass part/partition handles into existing functions in place of set handles, and have them interpreted specially if they happen to be part/partition handles? Mark, that wouldn't force you to actually represent parts/partitions as sets; you could interpret set handles in a specific range as part/partition ids. This would make the question of efficient booleans on the results completely separable from the parallel interface discussion, as well as allowing us to use our current function set dealing with entity sets to also apply to parts/partitions. - tim Mark Shephard wrote:
Tim,
No that is not true!!! The questions asked by an interface do not require specific information be stored. It requires the ability to answer the question!!!!! We very much use a partition model without sets or explicit storage of reverse classification. Reasons for this are explained in various papers. However, knowing that is less the issue. The reason for interface API is to allow people to answer the questions, not force them to use specific structures.
Mark
Tim Tautges wrote:
Then why do we need functions operating on parts or partitions at all? Those are reverse classification-type queries, right?
- tim
Mark Shephard wrote:
Since the first thing that was agreed to is that there would be the ability to deal parts that does not require an implementation to do things via sets (reverse classification), I sure hope we are not getting this issue confused again. As discussed in some technical detail at the boot camp, there are two high level approaches one based on classification (no use of sets) and reverse classification (what sets do well). All the SCOREC tools that are being used for parallel adaptive simulations, and (in the past) for parallel mesh generation, are based on a classification approach.
Onkar Sahni wrote:
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind.
If "sets as both parts and partitions" mean using entity-sets then we have already discussed this in great detail. I think the question in discussion is irrespective of how one implements parts and partitions? If I understood the question right then we need to answer queries which involve a specific part and a specific entity-set (where entity-set may span more than one part, may be local to a process). Now, if it is implied that choosing entity-sets for both parts and partitions allows to use set operations (booleans), outside the interface explicitly by applications, to answer such queries well then applications might as well do it in other equivalent ways (for example, loop over entities in set and ask residence parts to get entities in the entity-sey belonging to a specific part).
- Onkar
-- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Argonne National Laboratory ([email protected]) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706
Tim, I have never talked "data model" I have always spoke of functionality and API functions and have always contended that I have no concern about ones internal data model. Much of the discussion that got all this stuff confused to begin with with what (instanses vs whatever vs whatever) is forcing too much details of implementation data models into the process of defining the API and its functions. As far as I am concerned we are not keeping an eye on providing actual functionalities that are of use. Our only concern is the interface functions and we have always responded to the discussion of that topic. Onkar, Ting and others here have provided documents on possible functions and will continue to respond to discussion of the API functions. We are not in a position to respond on how others that use various internal representation approaches address their implementation (much of the recent discussion). We can say what we can implement and can iterate with people on changes to the functions such that we get a set that works for all. Mark Tim Tautges wrote:
But the language and constructs used to answer the questions (and to ask the questions) is what we call the data model. For example, the data model says we pass references to entities back and forth using handles. As proposed by you, the data model for parts refers to them using integer ids. I think those ids could just as easily be handles, and your implementation could implement those handles as integers.
As for how those are actually represented underneath, I strongly agree we shouldn't force implementations to use one thing or another. I'm just saying the handles used to refer to parts and partitions should also be usable in the other functions that deal with sets. I wasn't very careful in making this distinction in my writeup I sent earlier.
So, to go back to my original question, and back to what we had agreed going out of the last bootcamp: why can't we pass part/partition handles into existing functions in place of set handles, and have them interpreted specially if they happen to be part/partition handles? Mark, that wouldn't force you to actually represent parts/partitions as sets; you could interpret set handles in a specific range as part/partition ids.
This would make the question of efficient booleans on the results completely separable from the parallel interface discussion, as well as allowing us to use our current function set dealing with entity sets to also apply to parts/partitions.
- tim
Mark Shephard wrote:
Tim,
No that is not true!!! The questions asked by an interface do not require specific information be stored. It requires the ability to answer the question!!!!! We very much use a partition model without sets or explicit storage of reverse classification. Reasons for this are explained in various papers. However, knowing that is less the issue. The reason for interface API is to allow people to answer the questions, not force them to use specific structures.
Mark
Tim Tautges wrote:
Then why do we need functions operating on parts or partitions at all? Those are reverse classification-type queries, right?
- tim
Mark Shephard wrote:
Since the first thing that was agreed to is that there would be the ability to deal parts that does not require an implementation to do things via sets (reverse classification), I sure hope we are not getting this issue confused again. As discussed in some technical detail at the boot camp, there are two high level approaches one based on classification (no use of sets) and reverse classification (what sets do well). All the SCOREC tools that are being used for parallel adaptive simulations, and (in the past) for parallel mesh generation, are based on a classification approach.
Onkar Sahni wrote:
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind.
If "sets as both parts and partitions" mean using entity-sets then we have already discussed this in great detail. I think the question in discussion is irrespective of how one implements parts and partitions? If I understood the question right then we need to answer queries which involve a specific part and a specific entity-set (where entity-set may span more than one part, may be local to a process). Now, if it is implied that choosing entity-sets for both parts and partitions allows to use set operations (booleans), outside the interface explicitly by applications, to answer such queries well then applications might as well do it in other equivalent ways (for example, loop over entities in set and ask residence parts to get entities in the entity-sey belonging to a specific part).
- Onkar
The data model as I refer to it isn't about internal representation, it's part of the interface spec (I assert that it precedes the interface, in fact). I use the term "data structure" to refer to internal representation. - tim Mark Shephard wrote:
Tim,
I have never talked "data model" I have always spoke of functionality and API functions and have always contended that I have no concern about ones internal data model. Much of the discussion that got all this stuff confused to begin with with what (instanses vs whatever vs whatever) is forcing too much details of implementation data models into the process of defining the API and its functions. As far as I am concerned we are not keeping an eye on providing actual functionalities that are of use.
Our only concern is the interface functions and we have always responded to the discussion of that topic. Onkar, Ting and others here have provided documents on possible functions and will continue to respond to discussion of the API functions. We are not in a position to respond on how others that use various internal representation approaches address their implementation (much of the recent discussion). We can say what we can implement and can iterate with people on changes to the functions such that we get a set that works for all.
Mark
Tim Tautges wrote:
But the language and constructs used to answer the questions (and to ask the questions) is what we call the data model. For example, the data model says we pass references to entities back and forth using handles. As proposed by you, the data model for parts refers to them using integer ids. I think those ids could just as easily be handles, and your implementation could implement those handles as integers.
As for how those are actually represented underneath, I strongly agree we shouldn't force implementations to use one thing or another. I'm just saying the handles used to refer to parts and partitions should also be usable in the other functions that deal with sets. I wasn't very careful in making this distinction in my writeup I sent earlier.
So, to go back to my original question, and back to what we had agreed going out of the last bootcamp: why can't we pass part/partition handles into existing functions in place of set handles, and have them interpreted specially if they happen to be part/partition handles? Mark, that wouldn't force you to actually represent parts/partitions as sets; you could interpret set handles in a specific range as part/partition ids.
This would make the question of efficient booleans on the results completely separable from the parallel interface discussion, as well as allowing us to use our current function set dealing with entity sets to also apply to parts/partitions.
- tim
Mark Shephard wrote:
Tim,
No that is not true!!! The questions asked by an interface do not require specific information be stored. It requires the ability to answer the question!!!!! We very much use a partition model without sets or explicit storage of reverse classification. Reasons for this are explained in various papers. However, knowing that is less the issue. The reason for interface API is to allow people to answer the questions, not force them to use specific structures.
Mark
Tim Tautges wrote:
Then why do we need functions operating on parts or partitions at all? Those are reverse classification-type queries, right?
- tim
Mark Shephard wrote:
Since the first thing that was agreed to is that there would be the ability to deal parts that does not require an implementation to do things via sets (reverse classification), I sure hope we are not getting this issue confused again. As discussed in some technical detail at the boot camp, there are two high level approaches one based on classification (no use of sets) and reverse classification (what sets do well). All the SCOREC tools that are being used for parallel adaptive simulations, and (in the past) for parallel mesh generation, are based on a classification approach.
Onkar Sahni wrote:
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind.
If "sets as both parts and partitions" mean using entity-sets then we have already discussed this in great detail. I think the question in discussion is irrespective of how one implements parts and partitions? If I understood the question right then we need to answer queries which involve a specific part and a specific entity-set (where entity-set may span more than one part, may be local to a process). Now, if it is implied that choosing entity-sets for both parts and partitions allows to use set operations (booleans), outside the interface explicitly by applications, to answer such queries well then applications might as well do it in other equivalent ways (for example, loop over entities in set and ask residence parts to get entities in the entity-sey belonging to a specific part).
- Onkar
-- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Argonne National Laboratory ([email protected]) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706
We agree to look at it differently. So long as we define interface functions we can all live with. Tim Tautges wrote:
The data model as I refer to it isn't about internal representation, it's part of the interface spec (I assert that it precedes the interface, in fact). I use the term "data structure" to refer to internal representation.
- tim
Mark Shephard wrote:
Tim,
I have never talked "data model" I have always spoke of functionality and API functions and have always contended that I have no concern about ones internal data model. Much of the discussion that got all this stuff confused to begin with with what (instanses vs whatever vs whatever) is forcing too much details of implementation data models into the process of defining the API and its functions. As far as I am concerned we are not keeping an eye on providing actual functionalities that are of use.
Our only concern is the interface functions and we have always responded to the discussion of that topic. Onkar, Ting and others here have provided documents on possible functions and will continue to respond to discussion of the API functions. We are not in a position to respond on how others that use various internal representation approaches address their implementation (much of the recent discussion). We can say what we can implement and can iterate with people on changes to the functions such that we get a set that works for all.
Mark
Tim Tautges wrote:
But the language and constructs used to answer the questions (and to ask the questions) is what we call the data model. For example, the data model says we pass references to entities back and forth using handles. As proposed by you, the data model for parts refers to them using integer ids. I think those ids could just as easily be handles, and your implementation could implement those handles as integers.
As for how those are actually represented underneath, I strongly agree we shouldn't force implementations to use one thing or another. I'm just saying the handles used to refer to parts and partitions should also be usable in the other functions that deal with sets. I wasn't very careful in making this distinction in my writeup I sent earlier.
So, to go back to my original question, and back to what we had agreed going out of the last bootcamp: why can't we pass part/partition handles into existing functions in place of set handles, and have them interpreted specially if they happen to be part/partition handles? Mark, that wouldn't force you to actually represent parts/partitions as sets; you could interpret set handles in a specific range as part/partition ids.
This would make the question of efficient booleans on the results completely separable from the parallel interface discussion, as well as allowing us to use our current function set dealing with entity sets to also apply to parts/partitions.
- tim
Mark Shephard wrote:
Tim,
No that is not true!!! The questions asked by an interface do not require specific information be stored. It requires the ability to answer the question!!!!! We very much use a partition model without sets or explicit storage of reverse classification. Reasons for this are explained in various papers. However, knowing that is less the issue. The reason for interface API is to allow people to answer the questions, not force them to use specific structures.
Mark
Tim Tautges wrote:
Then why do we need functions operating on parts or partitions at all? Those are reverse classification-type queries, right?
- tim
Mark Shephard wrote:
Since the first thing that was agreed to is that there would be the ability to deal parts that does not require an implementation to do things via sets (reverse classification), I sure hope we are not getting this issue confused again. As discussed in some technical detail at the boot camp, there are two high level approaches one based on classification (no use of sets) and reverse classification (what sets do well). All the SCOREC tools that are being used for parallel adaptive simulations, and (in the past) for parallel mesh generation, are based on a classification approach.
Onkar Sahni wrote:
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind.
If "sets as both parts and partitions" mean using entity-sets then we have already discussed this in great detail. I think the question in discussion is irrespective of how one implements parts and partitions? If I understood the question right then we need to answer queries which involve a specific part and a specific entity-set (where entity-set may span more than one part, may be local to a process). Now, if it is implied that choosing entity-sets for both parts and partitions allows to use set operations (booleans), outside the interface explicitly by applications, to answer such queries well then applications might as well do it in other equivalent ways (for example, loop over entities in set and ask residence parts to get entities in the entity-sey belonging to a specific part).
- Onkar
But the language and constructs used to answer the questions (and to ask the questions) is what we call the data model. For example, the data model says we pass references to entities back and forth using handles. As proposed by you, the data model for parts refers to them using integer ids. I think those ids could just as easily be handles, and your implementation could implement those handles as integers.
We never proposed that "the data model for parts refers to them using integer ids". We don't do that in our existing functional implementation. There was a requirement that we need unique identifiers for parts (which is unrelated to part-handle or how it is implemented). There was a specific question during ITAPS bootcamp at RPI, that how do we take care of identifiers for parts and answer was we currently use integers and plan to use pair or combination of integers (for cases with multiple parts per process/task of parallel application).
So, to go back to my original question, and back to what we had agreed going out of the last bootcamp: why can't we pass part/partition handles into existing functions in place of set handles, and have them interpreted specially if they happen to be part/partition handles? Mark, that wouldn't force you to actually represent parts/partitions as sets; you could interpret set handles in a specific range as part/partition ids.
We have iterated that in mesh-specific functions under iMesh like ones Carl discussed in his proposal, getNumOfType, getNumOfTopo, getAllVtxCoords, getVtxCoordIndex, getEntities (and not entity-set specific functions like booleans on sets) we would allow to pass in part- and partition-handles (in place of entity-set handle).
This would make the question of efficient booleans on the results completely separable from the parallel interface discussion, as well as allowing us to use our current function set dealing with entity sets to also apply to parts/partitions.
Question under discussion is not on efficient booleans (for entity-sets). Our current function set (for parallel interface) doesn't allow for what we were trying to address. - Onkar
Onkar Sahni wrote:
But the language and constructs used to answer the questions (and to ask the questions) is what we call the data model. For example, the data model says we pass references to entities back and forth using handles. As proposed by you, the data model for parts refers to them using integer ids. I think those ids could just as easily be handles, and your implementation could implement those handles as integers.
We never proposed that "the data model for parts refers to them using integer ids". We don't do that in our existing functional implementation. There was a requirement that we need unique identifiers for parts (which is unrelated to part-handle or how it is implemented). There was a specific question during ITAPS bootcamp at RPI, that how do we take care of identifiers for parts and answer was we currently use integers and plan to use pair or combination of integers (for cases with multiple parts per process/task of parallel application)
Ok, my mistake, looking back at the pmodel document you do talk about iProcPart instances (which at this point are more like handles, I think?) .
So, to go back to my original question, and back to what we had agreed going out of the last bootcamp: why can't we pass part/partition handles into existing functions in place of set handles, and have them interpreted specially if they happen to be part/partition handles? Mark, that wouldn't force you to actually represent parts/partitions as sets; you could interpret set handles in a specific range as part/partition ids.
We have iterated that in mesh-specific functions under iMesh like ones Carl discussed in his proposal, getNumOfType, getNumOfTopo, getAllVtxCoords, getVtxCoordIndex, getEntities (and not entity-set specific functions like booleans on sets) we would allow to pass in part- and partition-handles (in place of entity-set handle).
This would make the question of efficient booleans on the results completely separable from the parallel interface discussion, as well as allowing us to use our current function set dealing with entity sets to also apply to parts/partitions.
Question under discussion is not on efficient booleans (for entity-sets). Our current function set (for parallel interface) doesn't allow for what we were trying to address.
The first two items from Karen's summary of the last phone call: - We discussed Carl's proposal to overload the mesh instance argument with partition instances and/or part handles to perform set operations. Jason expressed concerns that the overloading would make implementing the multiplexer difficult (at best), and would complicate the implementations as well. - Lori proposed adding partition information to the argument lists of the existing iMesh set-based functions. We agreed that we could change the iMesh interface, adding partition instance and part handle arguments to needed functions. Serial implementations could pass NULL for these arguments. I'm taking "set operations" to mean booleans, among other things, and the efficiency of booleans in the application is, I think, what started this conversation. Am I wrong? - tim
- Onkar
-- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Argonne National Laboratory ([email protected]) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706
The first two items from Karen's summary of the last phone call:
- We discussed Carl's proposal to overload the mesh instance argument with partition instances and/or part handles to perform set operations. Jason expressed concerns that the overloading would make implementing the multiplexer difficult (at best), and would complicate the implementations as well.
I think Carl's proposal discussed mesh-specific operations like getNumOfType, getNumOfTopo, getAllVtxCoords, getVtxCoordIndex, getEntities and not "set-operations", as I understand it. Can we deal with overloading (considering multiplexer too), if we talk in terms of mesh instances, partition instances and part instances (not part-handles). The only difference I have learned in all this time between "handles" and "instances" under ITAPS terminology is that handles can be id- or index- based (whereas instance cannot be). Actually in Carl's proposal everything was handle, i.e., mesh-handle, partition-handle and part-handle, see below. / partition handle getNumOfType| mesh instance , root set handle , type, result, err ) \ part handle entity set handle I think we didn't made it clear or may be it got lost in details, that we can easily handle (with no complication in our implementation) what Carl has proposed. If overloading makes multiplexer difficult (at best) only, and not impossible, then I think we should see how much difficulty are we talking about (may be compare to effort and usability of changing the existing iMesh interface). As Mark S. said we are "forcing too much details of implementation data models into the process of defining the API and its functions".
I'm taking "set operations" to mean booleans, among other things, and the efficiency of booleans in the application is, I think, what started this conversation. Am I wrong?
I think we were not worrying about efficiency at this time. We wanted to address the requirements first (through interface). - Onkar
Tim Tautges wrote:
Onkar Sahni wrote:
But the language and constructs used to answer the questions (and to ask the questions) is what we call the data model. For example, the data model says we pass references to entities back and forth using handles. As proposed by you, the data model for parts refers to them using integer ids. I think those ids could just as easily be handles, and your implementation could implement those handles as integers.
We never proposed that "the data model for parts refers to them using integer ids". We don't do that in our existing functional implementation. There was a requirement that we need unique identifiers for parts (which is unrelated to part-handle or how it is implemented). There was a specific question during ITAPS bootcamp at RPI, that how do we take care of identifiers for parts and answer was we currently use integers and plan to use pair or combination of integers (for cases with multiple parts per process/task of parallel application)
Ok, my mistake, looking back at the pmodel document you do talk about iProcPart instances (which at this point are more like handles, I think?)
.
So, to go back to my original question, and back to what we had agreed going out of the last bootcamp: why can't we pass part/partition handles into existing functions in place of set handles, and have them interpreted specially if they happen to be part/partition handles? Mark, that wouldn't force you to actually represent parts/partitions as sets; you could interpret set handles in a specific range as part/partition ids.
We have iterated that in mesh-specific functions under iMesh like ones Carl discussed in his proposal, getNumOfType, getNumOfTopo, getAllVtxCoords, getVtxCoordIndex, getEntities (and not entity-set specific functions like booleans on sets) we would allow to pass in part- and partition-handles (in place of entity-set handle).
This would make the question of efficient booleans on the results completely separable from the parallel interface discussion, as well as allowing us to use our current function set dealing with entity sets to also apply to parts/partitions.
Question under discussion is not on efficient booleans (for entity-sets). Our current function set (for parallel interface) doesn't allow for what we were trying to address.
The first two items from Karen's summary of the last phone call:
- We discussed Carl's proposal to overload the mesh instance argument with partition instances and/or part handles to perform set operations. Jason expressed concerns that the overloading would make implementing the multiplexer difficult (at best), and would complicate the implementations as well.
- Lori proposed adding partition information to the argument lists of the existing iMesh set-based functions. We agreed that we could change the iMesh interface, adding partition instance and part handle arguments to needed functions. Serial implementations could pass NULL for these arguments.
I'm taking "set operations" to mean booleans, among other things, and the efficiency of booleans in the application is, I think, what started this conversation. Am I wrong?
No, what started the whole conversation was the need to support questions like, "How many triangles are there in some set across the entire partition?" and "What are all regions in some set in a particular part?" The proposal I sent out was to overload parts, mesh instances, and partitions as the first argument in existing iMesh functions. This was shot down on the basis that it makes multiplexing difficult, at least, and perhaps impossible. The alternative on which we (I think) more or less reached consensus at the telecon was to expand the argument lists for relevant functions to include a partition handle and a part handle. For example, iMesh_getNumOfType(iMesh_instance instance, const iBase_EntitySetHandle entity_set_handle, const PartitionHandle partition_handle, const PartHandle part_handle, const int entity_type, int *num_type, int *err); Now, I've just done a pass through the iMesh interface to make a list of what functions are likely to need this treatment, and here's what I've come up with: Need part, instance, and partition versions: getNumOfType, getNumOfTopo, getNumEntSets* The asterisk on getNumEntSets is for partitions: we don't currently have defined semantics for identifying logical sets that are spread across parts as being one set. Given that, it isn't clear to me whether asking for a partition-wide number of entity sets necessarily gives useful information. This same logic motivates me to leave all of the set stuff out, too, on the basis that sets will (under the current paradigm) intrinsically are native to an instance (not a part, unless we make a change there...), and so talking about adding one set to another on a part (for instance) doesn't make sense. Any thoughts? (Tim, I'm bracing myself... :-)) Need part and instance versions, but -not- partition, in keeping with our stated aim of not allowing global results that will be absurdly / unscalably large: getAllVtxCoords, getVtxCoordIndex, getAdjEntities, getEntities (get2ndAdjEntities will go here, too, once it's added officially) Need part and instance versions, but -not- partition (IMO), because the parallel iteration is again intrinsically unscalable. initEntArrIter, initEntIter So the bottom line is that we're looking at, if my count is correct, no more than ten functions that will need to be modified in any way. If that small number turns out to really be correct in everyone else's view, then it's a manageable enough number that it isn't absolutely anathema to -duplicate- a function or two so that the serial interface doesn't have to carry around a bunch of null handles as args (or maybe we could add a macro to expand something like the current serial version to the full version with extra args). Carl -- ------------------------------------------------------------------------ Dr. Carl Ollivier-Gooch, P.Eng. Voice: +1-604-822-1854 Associate Professor Fax: +1-604-822-2403 Department of Mechanical Engineering email: [email protected] University of British Columbia http://www.mech.ubc.ca/~cfog Vancouver, BC V6T 1Z4 http://tetra.mech.ubc.ca/ANSLab/ ------------------------------------------------------------------------
So the bottom line is that we're looking at, if my count is correct, no more than ten functions that will need to be modified in any way. If that small number turns out to really be correct in everyone else's view, then it's a manageable enough number that it isn't absolutely anathema to -duplicate- a function or two so that the serial interface doesn't have to carry around a bunch of null handles as args (or maybe we could add a macro to expand something like the current serial version to the full version with extra args).
I agree with Carl, a bunch of extra handles (that may be null most of the time) will make interface confusing, likely to do errors and burdensome (in using it, learning it etc.) and it will not be a good idea. Without extra part (and partition) handles will likely keep, serial and parallel portions, clean and separate. - Onkar
Carl Ollivier-Gooch wrote:
Tim Tautges wrote:
The first two items from Karen's summary of the last phone call:
- We discussed Carl's proposal to overload the mesh instance argument with partition instances and/or part handles to perform set operations. Jason expressed concerns that the overloading would make implementing the multiplexer difficult (at best), and would complicate the implementations as well.
- Lori proposed adding partition information to the argument lists of the existing iMesh set-based functions. We agreed that we could change the iMesh interface, adding partition instance and part handle arguments to needed functions. Serial implementations could pass NULL for these arguments.
I'm taking "set operations" to mean booleans, among other things, and the efficiency of booleans in the application is, I think, what started this conversation. Am I wrong?
No, what started the whole conversation was the need to support questions like, "How many triangles are there in some set across the entire partition?" and "What are all regions in some set in a particular part?"
Well ok, if you want to get that specific, then yes. But the second of those is certainly a set operation (not necessarily entity sets as in the iMesh interface, just intersection/etc. between different lists of entities). And the two questions above are different in the sense that one might involve parallel communication and the other doesn't. So I count three fundamental questions we're passing around in one discussion here: 1) Do we want to do booleans on query results below the interface level? An example query would be "What are all regions in some set in a particular part?" 2) How do we express and query sets spread across processors? 3) How do we refer to parts and partitions in the interface, and which functions do we use to get information about them? I think we need to sort out these questions at the conceptual level before getting into specific functions. I also think they are separable questions and should be treated that way first. Thoughts? - tim
The proposal I sent out was to overload parts, mesh instances, and partitions as the first argument in existing iMesh functions. This was shot down on the basis that it makes multiplexing difficult, at least, and perhaps impossible.
The alternative on which we (I think) more or less reached consensus at the telecon was to expand the argument lists for relevant functions to include a partition handle and a part handle. For example,
iMesh_getNumOfType(iMesh_instance instance, const iBase_EntitySetHandle entity_set_handle, const PartitionHandle partition_handle, const PartHandle part_handle, const int entity_type, int *num_type, int *err);
Now, I've just done a pass through the iMesh interface to make a list of what functions are likely to need this treatment, and here's what I've come up with:
Need part, instance, and partition versions: getNumOfType, getNumOfTopo, getNumEntSets*
The asterisk on getNumEntSets is for partitions: we don't currently have defined semantics for identifying logical sets that are spread across parts as being one set. Given that, it isn't clear to me whether asking for a partition-wide number of entity sets necessarily gives useful information. This same logic motivates me to leave all of the set stuff out, too, on the basis that sets will (under the current paradigm) intrinsically are native to an instance (not a part, unless we make a change there...), and so talking about adding one set to another on a part (for instance) doesn't make sense. Any thoughts? (Tim, I'm bracing myself... :-))
Need part and instance versions, but -not- partition, in keeping with our stated aim of not allowing global results that will be absurdly / unscalably large: getAllVtxCoords, getVtxCoordIndex, getAdjEntities, getEntities (get2ndAdjEntities will go here, too, once it's added officially)
Need part and instance versions, but -not- partition (IMO), because the parallel iteration is again intrinsically unscalable. initEntArrIter, initEntIter
So the bottom line is that we're looking at, if my count is correct, no more than ten functions that will need to be modified in any way. If that small number turns out to really be correct in everyone else's view, then it's a manageable enough number that it isn't absolutely anathema to -duplicate- a function or two so that the serial interface doesn't have to carry around a bunch of null handles as args (or maybe we could add a macro to expand something like the current serial version to the full version with extra args).
Carl
-- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Argonne National Laboratory ([email protected]) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706
Well ok, if you want to get that specific, then yes. But the second of those is certainly a set operation (not necessarily entity sets as in the iMesh interface, just intersection/etc. between different lists of entities). And the two questions above are different in the sense that one might involve parallel communication and the other doesn't.
So I count three fundamental questions we're passing around in one discussion here:
1) Do we want to do booleans on query results below the interface level? An example query would be "What are all regions in some set in a particular part?"
We would not even do entity-set boolean (and produce a new entity-set) below the interface for such a query. Mostly we will try: given some set, loop over set, ask each entity in set which part it belongs to.
2) How do we express and query sets spread across processors?
We agreed that this would require communications and mostly will be done by application outside/above interface.
3) How do we refer to parts and partitions in the interface, and which functions do we use to get information about them?
We refer them as handles and/or instances, like part-handle (but not entity-sets). - Onkar
I think we need to sort out these questions at the conceptual level before getting into specific functions. I also think they are separable questions and should be treated that way first. Thoughts?
- tim
The proposal I sent out was to overload parts, mesh instances, and partitions as the first argument in existing iMesh functions. This was shot down on the basis that it makes multiplexing difficult, at least, and perhaps impossible.
The alternative on which we (I think) more or less reached consensus at the telecon was to expand the argument lists for relevant functions to include a partition handle and a part handle. For example,
iMesh_getNumOfType(iMesh_instance instance, const iBase_EntitySetHandle entity_set_handle, const PartitionHandle partition_handle, const PartHandle part_handle, const int entity_type, int *num_type, int *err);
Now, I've just done a pass through the iMesh interface to make a list of what functions are likely to need this treatment, and here's what I've come up with:
Need part, instance, and partition versions: getNumOfType, getNumOfTopo, getNumEntSets*
The asterisk on getNumEntSets is for partitions: we don't currently have defined semantics for identifying logical sets that are spread across parts as being one set. Given that, it isn't clear to me whether asking for a partition-wide number of entity sets necessarily gives useful information. This same logic motivates me to leave all of the set stuff out, too, on the basis that sets will (under the current paradigm) intrinsically are native to an instance (not a part, unless we make a change there...), and so talking about adding one set to another on a part (for instance) doesn't make sense. Any thoughts? (Tim, I'm bracing myself... :-))
Need part and instance versions, but -not- partition, in keeping with our stated aim of not allowing global results that will be absurdly / unscalably large: getAllVtxCoords, getVtxCoordIndex, getAdjEntities, getEntities (get2ndAdjEntities will go here, too, once it's added officially)
Need part and instance versions, but -not- partition (IMO), because the parallel iteration is again intrinsically unscalable. initEntArrIter, initEntIter
So the bottom line is that we're looking at, if my count is correct, no more than ten functions that will need to be modified in any way. If that small number turns out to really be correct in everyone else's view, then it's a manageable enough number that it isn't absolutely anathema to -duplicate- a function or two so that the serial interface doesn't have to carry around a bunch of null handles as args (or maybe we could add a macro to expand something like the current serial version to the full version with extra args).
Carl
-- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3
Tim Tautges Argonne National Laboratory ([email protected]) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706
I've only seen comments from Onkar on the question below, and while useful, they were more like answers to the questions. I'm really interested in nailing down exactly what the issues are we're trying to address with this interface extension discussion (the specific discussion about Carl's proposal). Does the silence mean everyone agrees that the three questions below summarize the issues well enough? Or are you all just tired of reading my diatribes? Or both? (Don't answer that...) - tim Tim Tautges wrote:
So I count three fundamental questions we're passing around in one discussion here:
1) Do we want to do booleans on query results below the interface level? An example query would be "What are all regions in some set in a particular part?"
2) How do we express and query sets spread across processors?
3) How do we refer to parts and partitions in the interface, and which functions do we use to get information about them?
I think we need to sort out these questions at the conceptual level before getting into specific functions. I also think they are separable questions and should be treated that way first. Thoughts?
- tim
-- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Argonne National Laboratory ([email protected]) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706
On the classification, reverse-classification issue, has anyone considered the utility of adding functions to iMesh like the following. The functions below basically implement the 'dual' nature of the classification, reverse-classification representation of key database content. In Silo, we do something similar with materials. Some applications define materials by defining a material 'value' for every 'zone' in the mesh (classification) and some do it by defining for each material, the zones that are contained in that material (reverse classification). We have a function in Silo that simply maps from one to the other as the needs of the application(s) using material information vary. I am wondering if we can basically add something similar to iMesh. Below is an albeit not very well thought out suggestion. If there is interest, its a starting point... The following function will create the a 'collection of sets' dual to a given entity-level tag defined over an array of entities. Calling this function has the effect of creating in the iMesh instance a collection of sets such that each set contains entities given in 'entity_handles' with a common tag value. The sets could be returned in the call (which I've chosen not to do here). Or, they could be queried back out from the root. In addition, some 'filtering' could be added to create sets for only some tag values, or to specify ranges of tag values (for non-discrete tags). void iMesh_createDualSetsFromTagValues(iMesh_Instance instance, /*in*/ const iBase_EntityHandle* entity_handles, /*in*/ const int entity_handles_size, /*in*/ const iBase_TagHandle tag_handle, /*out*/ int *err); The following function does the 'opposite' of the above. That is, it creates an entity-level tag over the set of entities represented by the union of all the entities in sets specified by the 'set_handles' array. It will create a different tag value for each set and then assign that value to all entities in the given set. You could imagine passing to iMesh the set of 'values' it should assign as well. void iMesh_createDualTagValuesFromSets(iMesh_Instance instance, /*in*/ const iBase_EntitySetHandle* set_handles, /*in*/ const int set_handles_size, /*in*/ const iBase_TagHandle tag_handle, /*out*/ int *err); Mark Mark Shephard wrote:
Since the first thing that was agreed to is that there would be the ability to deal parts that does not require an implementation to do things via sets (reverse classification), I sure hope we are not getting this issue confused again. As discussed in some technical detail at the boot camp, there are two high level approaches one based on classification (no use of sets) and reverse classification (what sets do well). All the SCOREC tools that are being used for parallel adaptive simulations, and (in the past) for parallel mesh generation, are based on a classification approach.
Onkar Sahni wrote:
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind.
If "sets as both parts and partitions" mean using entity-sets then we have already discussed this in great detail. I think the question in discussion is irrespective of how one implements parts and partitions? If I understood the question right then we need to answer queries which involve a specific part and a specific entity-set (where entity-set may span more than one part, may be local to a process). Now, if it is implied that choosing entity-sets for both parts and partitions allows to use set operations (booleans), outside the interface explicitly by applications, to answer such queries well then applications might as well do it in other equivalent ways (for example, loop over entities in set and ask residence parts to get entities in the entity-sey belonging to a specific part).
- Onkar
-- Mark C. Miller, Lawrence Livermore National Laboratory email: mailto:[email protected] (M/T/W) (925)-423-5901 (!!LLNL BUSINESS ONLY!!) (Th/F) (530)-753-8511 (!!LLNL BUSINESS ONLY!!)
Mark, The short answer to your question on defining iMesh functions to map between the two forms have been included in iMesh. I agree that having something for that would be useful. To the best of my knowledge, most applications tend to rely on having one or the other as their base form. Thus interoperability of applications could benefit from the ability to go between the two. In addition, there may be times where one is fully one way and wants some specific aspects in the other. As an example, in the case of a classification based approach, it could be some specific reverse classification information that is desired for a few specific things one may want to define and store the reverse classification sets for (e.g., the mesh faces on a model face that has a natural boundary condition.) Note that in the particular formalism of classification and reverse classification the RPI group uses, each mesh entity maintains classification against the single lowest order model entity it is on, and reverse classification is specifically the set of equal order mesh entities classified on a model entity (adjacency information can quickly tell you any other information). Since iMesh does not have a stated formalism of classification and reverse classification, I do not know how easy it would be to really add the types of functions you mention to iMesh. Obviously the RPI group would be supportive of supporting such things. However, I expect there would be some objections to doing that. Mark Mark Miller wrote:
On the classification, reverse-classification issue, has anyone considered the utility of adding functions to iMesh like the following.
The functions below basically implement the 'dual' nature of the classification, reverse-classification representation of key database content. In Silo, we do something similar with materials. Some applications define materials by defining a material 'value' for every 'zone' in the mesh (classification) and some do it by defining for each material, the zones that are contained in that material (reverse classification). We have a function in Silo that simply maps from one to the other as the needs of the application(s) using material information vary. I am wondering if we can basically add something similar to iMesh. Below is an albeit not very well thought out suggestion. If there is interest, its a starting point...
The following function will create the a 'collection of sets' dual to a given entity-level tag defined over an array of entities. Calling this function has the effect of creating in the iMesh instance a collection of sets such that each set contains entities given in 'entity_handles' with a common tag value. The sets could be returned in the call (which I've chosen not to do here). Or, they could be queried back out from the root. In addition, some 'filtering' could be added to create sets for only some tag values, or to specify ranges of tag values (for non-discrete tags).
void iMesh_createDualSetsFromTagValues(iMesh_Instance instance,
/*in*/ const iBase_EntityHandle* entity_handles,
/*in*/ const int entity_handles_size,
/*in*/ const iBase_TagHandle tag_handle,
/*out*/ int *err);
The following function does the 'opposite' of the above. That is, it creates an entity-level tag over the set of entities represented by the union of all the entities in sets specified by the 'set_handles' array. It will create a different tag value for each set and then assign that value to all entities in the given set. You could imagine passing to iMesh the set of 'values' it should assign as well.
void iMesh_createDualTagValuesFromSets(iMesh_Instance instance,
/*in*/ const iBase_EntitySetHandle* set_handles,
/*in*/ const int set_handles_size,
/*in*/ const iBase_TagHandle tag_handle,
/*out*/ int *err);
Mark
Mark Shephard wrote:
Since the first thing that was agreed to is that there would be the ability to deal parts that does not require an implementation to do things via sets (reverse classification), I sure hope we are not getting this issue confused again. As discussed in some technical detail at the boot camp, there are two high level approaches one based on classification (no use of sets) and reverse classification (what sets do well). All the SCOREC tools that are being used for parallel adaptive simulations, and (in the past) for parallel mesh generation, are based on a classification approach.
Onkar Sahni wrote:
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind. If "sets as both parts and partitions" mean using entity-sets then we have already discussed this in great detail. I think the question in discussion is irrespective of how one implements parts and partitions? If I understood the question right then we need to answer queries which involve a specific part and a specific entity-set (where entity-set may span more than one part, may be local to a process). Now, if it is implied that choosing entity-sets for both parts and partitions allows to use set operations (booleans), outside the interface explicitly by applications, to answer such queries well then applications might as well do it in other equivalent ways (for example, loop over entities in set and ask residence parts to get entities in the entity-sey belonging to a specific part).
- Onkar
-- Mark C. Miller, Lawrence Livermore National Laboratory email: mailto:[email protected] (M/T/W) (925)-423-5901 (!!LLNL BUSINESS ONLY!!) (Th/F) (530)-753-8511 (!!LLNL BUSINESS ONLY!!)
Onkar Sahni wrote:
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind.
If "sets as both parts and partitions" mean using entity-sets then we have already discussed this in great detail. I think the question in discussion is irrespective of how one implements parts and partitions? If I understood the question right then we need to answer queries which involve a specific part and a specific entity-set (where entity-set may span more than one part, may be local to a process). Now, if it is implied that choosing entity-sets for both parts and partitions allows to use set operations (booleans), outside the interface explicitly by applications, to answer such queries well then applications might as well do it in other equivalent ways (for example, loop over entities in set and ask residence parts to get entities in the entity-sey belonging to a specific part).
Sure, but the implication in the current discussion is that doing booleans in the applications themselves is undesirable. Therefore we are looking for a way to represent these questions in the interface. To that I assert that there are lots of other set booleans that are also likely and possibly more common, and therefore we should design these functions to be more general than just applying to parts and partitions. Examples of these queries include: - boundary condition faces which are owned locally - vertices in a set being smoothed which are also on a model face - elements at one refinement level with faces on the skin - tim
- Onkar
-- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Argonne National Laboratory ([email protected]) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind.
If "sets as both parts and partitions" mean using entity-sets then we have already discussed this in great detail. I think the question in discussion is irrespective of how one implements parts and partitions? If I understood the question right then we need to answer queries which involve a specific part and a specific entity-set (where entity-set may span more than one part, may be local to a process). Now, if it is implied that choosing entity-sets for both parts and partitions allows to use set operations (booleans), outside the interface explicitly by applications, to answer such queries well then applications might as well do it in other equivalent ways (for example, loop over entities in set and ask residence parts to get entities in the entity-sey belonging to a specific part).
Sure, but the implication in the current discussion is that doing booleans in the applications themselves is undesirable. Therefore we are looking for a way to represent these questions in the interface. To that I assert that there are lots of other set booleans that are also likely and possibly more common, and therefore we should design these functions to be more general than just applying to parts and partitions. Examples of these queries include: - boundary condition faces which are owned locally - vertices in a set being smoothed which are also on a model face - elements at one refinement level with faces on the skin
These general queries are not (at least to me) related to "core needs that prevent us from using sets as both parts and partitions". Infact, applications can also ask for intersection (i.e., this+this+that) of all examples queries listed above. I would consider these more "utility" type functions which I hope is not confused with parallel interface ("parts and partitions") and may even be separate form mesh interface (iMesh, serial one) as one can ask these general questions at geometric model level too (iGeom)? Do we use "entity-sets" in iGeom (sorry, I am not familiar with iGeom)? For "utility" type functions, we can easily deal with them by following similar concepts as STL algorithms, like find, sort etc., that take iterators and some operators (may be we can pass in iMesh entity iterators and some iMesh boolean operators). For example, numBCFacesOwned = count(bcFaceIterBegin,bcFaceIterEnd,&iPart_isEntOwned); - Onkar
Onkar Sahni wrote:
So again, I go back to asking: what are the core needs that prevent us from using sets as both parts and partitions? The entity set mechanism was designed with this specific usage in mind. If "sets as both parts and partitions" mean using entity-sets then we have already discussed this in great detail. I think the question in discussion is irrespective of how one implements parts and partitions? If I understood the question right then we need to answer queries which involve a specific part and a specific entity-set (where entity-set may span more than one part, may be local to a process). Now, if it is implied that choosing entity-sets for both parts and partitions allows to use set operations (booleans), outside the interface explicitly by applications, to answer such queries well then applications might as well do it in other equivalent ways (for example, loop over entities in set and ask residence parts to get entities in the entity-sey belonging to a specific part).
Sure, but the implication in the current discussion is that doing booleans in the applications themselves is undesirable. Therefore we are looking for a way to represent these questions in the interface. To that I assert that there are lots of other set booleans that are also likely and possibly more common, and therefore we should design these functions to be more general than just applying to parts and partitions. Examples of these queries include: - boundary condition faces which are owned locally - vertices in a set being smoothed which are also on a model face - elements at one refinement level with faces on the skin
These general queries are not (at least to me) related to "core needs that prevent us from using sets as both parts and partitions".
No, but I'm not sure the part-set booleans under discussion are either (up to now it's been more of an efficiency issue, hasn't it?)
Infact, applications can also ask for intersection (i.e., this+this+that) of all examples queries listed above. I would consider these more "utility" type functions which I hope is not confused with parallel interface ("parts and partitions") and may even be separate form mesh interface (iMesh, serial one) as one can ask these general questions at geometric model level too (iGeom)? Do we use "entity-sets" in iGeom (sorry, I am not familiar with iGeom)?
In the original SIDL interface, entity sets were part of iBase, inherited by both iMesh and iGeom. In that sense, I would expect any extra set functions to also appear in iGeom.
For "utility" type functions, we can easily deal with them by following similar concepts as STL algorithms, like find, sort etc., that take iterators and some operators (may be we can pass in iMesh entity iterators and some iMesh boolean operators). For example,
numBCFacesOwned = count(bcFaceIterBegin,bcFaceIterEnd,&iPart_isEntOwned);
I'm not sure that would work very well, depending on the cost of calls through the interface (I know for a fact a SIDL-based interface like that would be way too expensive). Note that the template-based STL stuff only works in C++ and not across languages, and that's an important class of applications. - tim
- Onkar
-- ================================================================ "You will keep in perfect peace him whose mind is steadfast, because he trusts in you." Isaiah 26:3 Tim Tautges Argonne National Laboratory ([email protected]) (telecommuting from UW-Madison) phone: (608) 263-8485 1500 Engineering Dr. fax: (608) 263-4499 Madison, WI 53706
participants (8)
-
Carl Ollivier-Gooch -
Devine, Karen D. -
Jason Kraftcheck -
Mark Miller -
Mark Shephard -
Onkar Sahni -
Tim Tautges -
Vitus Leung