December 2021 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] mpi_comm_create_group()
by Adrian Croucher 02 Dec '21

02 Dec '21

On 12/3/21 5:31 PM, Jed Brown wrote: > You'd need more calls in case of overlap (so MPI_Comm_create_group is more natural), but non-participating ranks is easy. > > | A process may supply the color value MPI_UNDEFINED, in which case newcomm returns MPI_COMM_NULL. Ah, interesting. I guess mpi_comm_split() might also have the advantage that each rank only needs to know if it is in the group or not, rather than needing an array of all participating ranks, as mpi_comm_create() and mpi_comm_create_group() do. - Adrian -- Dr Adrian Croucher Senior Research Fellow Department of Engineering Science University of Auckland, New Zealand email: a.croucher(a)auckland.ac.nz tel: +64 (0)9 923 4611

1 0

Re: [petsc-dev] mpi_comm_create_group()
by Jed Brown 02 Dec '21

02 Dec '21

Adrian Croucher <a.croucher(a)auckland.ac.nz> writes: > On 12/3/21 12:09 PM, Jed Brown wrote: >> Many find MPI_Comm_split() to be a more convenient interface. > Yes, I don't think that will work for my case though, as my groups may > overlap, and some processes may not be in any group. You'd need more calls in case of overlap (so MPI_Comm_create_group is more natural), but non-participating ranks is easy. | A process may supply the color value MPI_UNDEFINED, in which case newcomm returns MPI_COMM_NULL.

1 0

Re: [petsc-dev] mpi_comm_create_group()
by Adrian Croucher 02 Dec '21

02 Dec '21

On 12/3/21 12:09 PM, Jed Brown wrote: > Many find MPI_Comm_split() to be a more convenient interface. Yes, I don't think that will work for my case though, as my groups may overlap, and some processes may not be in any group. > > I think the man page is fairly clear. > > | MPI_Comm_create_group is similar to MPI_Comm_create; however, > MPI_Comm_create must be called by all processes in the group of comm, > whereas MPI_Comm_create_group must be called by all processes in > group, which is a subgroup of the group of comm. Yes, it looks like the mpitutorial.com example I followed was a bit misleading then. Thanks! - Adrian -- Dr Adrian Croucher Senior Research Fellow Department of Engineering Science University of Auckland, New Zealand email: a.croucher(a)auckland.ac.nz tel: +64 (0)9 923 4611

1 0

Re: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot?
by Hamilton, Aidan 02 Dec '21

02 Dec '21

Barry, That is exactly the case, I only have work dofs for passing flux variables. The variables that actually matter for the solution are only on the edges. Works completely correctly in that case as far as I can tell. Your example is very interesting. Is this intended to be fixed so that DMNetwork can handle these cases or is not an intended usage? Even for our case with traffic flow it would be very convenient to be handle multi-edged graphs. Best, Aidan From: Barry Smith <bsmith(a)petsc.dev> Date: Thursday, December 2, 2021 at 5:07 PM To: Zhang, Hong <hzhang(a)mcs.anl.gov> Cc: For users of the development version of PETSc <petsc-dev(a)mcs.anl.gov>, Sundermann, Rylee Edwin - SDSU Student <rylee.sundermann(a)jacks.sdstate.edu>, Hamilton, Aidan <ahamilton(a)anl.gov> Subject: Re: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot? Aiden, Do you have true degrees of freedom on both the edges and the vertices? If only have edge variables and no vertex variables but only use "work" values on vertices than maybe DMNETWORK might behave as you need. But my trivial example defintely shows a problem with only edge vertex variables. The DMPLEX object believes there are more than two vertices when there are only two because it cannot determine that some vertices are actually the same. Barry On Dec 2, 2021, at 5:31 PM, Zhang, Hong via petsc-dev <petsc-dev(a)mcs.anl.gov<mailto:[email protected]>> wrote: All, I'm forwarding Aidan's comments on this issue (see below). Aidan has done extensive study on DMPlex and has been working on hyperbolic flow applications. Satish: Please add Aidan to petsc-dev mailing list. Hong ________________________________ From: Aidan Hamilton <aidan(a)udel.edu<mailto:[email protected]>> Sent: Thursday, December 2, 2021 2:34 PM To: Zhang, Hong <hzhang(a)mcs.anl.gov<mailto:[email protected]>> Cc: Sundermann, Rylee Edwin - SDSU Student <rylee.sundermann(a)jacks.sdstate.edu<mailto:[email protected]>> Subject: Re: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot? Hong, I am not on petsc-dev, I guess I should be. Also you can push, you’re always free to push changes. And it doesn’t matter for anything on my end. Each edge is distinct topological entity, so I have the directed graph V0 ---- E0 -• V1 ----- E1 ----• V0 I don’t need any special identification. And it works in parallel with globaltolocal seemingly working . Now this is not a standard graph, it follows the more general notion of directed graph, see Diestel page 25, that allows for multi-edges. Now for my case I don’t need to distinguish between the two edges at all beyond each one having their own unique edge id. In all of our use cases for traffic I don’t think we need to care (and honestly I’m not sure if Shri actually needs to care either). Now we do have the topological distinction of direction for all of our graphs, which we can use to assign direction to traffic flow. And that will always work. And this notion of direction is the only distinction we need to care about, and we have it. It’s worth noting that a CW-Complex does support this notion of multiedges, no reason for it not to, so DMPlex should be supporting its topological operations with this case in mind. If DMplex can’t support this then dmplex isn’t doing what it claims it does. As long as you as the user know that you are dealing with a multi-edge graph, I don’t see why there would be any issues. Any topological query of a multi-edge graph should behave as you would expect a multi-edge graph to behave, though it may behave differently than single edge graphs, as long as you are aware of the difference when you code your residuals and other functions on dmnetwork it should be fine. Best, Aidan From: Zhang, Hong <hzhang(a)mcs.anl.gov<mailto:[email protected]>> Date: Thursday, December 2, 2021 at 1:56 PM To: Aidan Hamilton <aidan(a)udel.edu<mailto:[email protected]>> Cc: Sundermann, Rylee Edwin - SDSU Student <rylee.sundermann(a)jacks.sdstate.edu<mailto:[email protected]>> Subject: Fw: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot? Aidan, Are you on petsc-dev email list? How do you distinguish two parallel edges connecting the same junctions in your traffic code? Do you use offset_vto, offset_vfrom in EdgeFE? I removed white spaces in dgnet.c and dgnet.h. Can I push the changes? Hong ________________________________ From: petsc-dev <petsc-dev-bounces(a)mcs.anl.gov<mailto:[email protected]>> on behalf of Barry Smith <bsmith(a)petsc.dev<mailto:[email protected]>> Sent: Wednesday, December 1, 2021 7:41 PM To: Abhyankar, Shrirang G <shrirang.abhyankar(a)pnnl.gov<mailto:[email protected]>> Cc: petsc-dev <petsc-dev(a)mcs.anl.gov<mailto:[email protected]>>; Betrie, Getnet <gbetrie(a)anl.gov<mailto:[email protected]>> Subject: Re: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot? Shri, If you provided your graph in parallel I suspect that the GlobalToLocal and LocalToGlobal would not work as you expect and put everything in the appropriate place. I'll set up a simple code that takes exactly Get's configuration and send it out so everyone can run it and see exactly what happens. Barry On Dec 1, 2021, at 8:00 PM, Abhyankar, Shrirang G via petsc-dev <petsc-dev(a)mcs.anl.gov<mailto:[email protected]>> wrote: From: Matthew Knepley <knepley(a)gmail.com<mailto:[email protected]>> Date: Wednesday, December 1, 2021 at 5:57 PM To: "Brown, Jedediah A (VISIT)" <jed(a)jedbrown.org<mailto:[email protected]>> Cc: "Abhyankar, Shrirang G" <shrirang.abhyankar(a)pnnl.gov<mailto:[email protected]>>, PETSc Development <petsc-dev(a)mcs.anl.gov<mailto:[email protected]>>, Getnet Betrie <gbetrie(a)anl.gov<mailto:[email protected]>> Subject: Re: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot? Check twice before you click! This email originated from outside PNNL. On Wed, Dec 1, 2021 at 6:55 PM Jed Brown <jed(a)jedbrown.org<mailto:[email protected]>> wrote: Matthew Knepley <knepley(a)gmail.com<mailto:[email protected]>> writes: > On Wed, Dec 1, 2021 at 5:17 PM Abhyankar, Shrirang G < > shrirang.abhyankar(a)pnnl.gov<mailto:[email protected]>> wrote: > >> “You can certainly have many fields on a given edge, but I don't know >> what it would mean to have two edges since no topological query could tell >> the difference.” >> >> >> >> The two edges in a power grid represent two parallel power lines that are >> connected between two locations (vertices). There are line ids (stored in >> the component data) to distinguish the two lines. >> > > Yes, so you can tell the difference in the function space (since difference > current passes down each one), but _topologically_ you cannot. If you put > duplicate cells in, then > some topological queries will give unexpected results, like the join of the > two vertices. This could be modeled with some ghost vertices. So instead of a ------ b \_____/ you would set up a ---o---- b \___o___/ Those ghost vertices don't have to "do" anything, but they make the edges topologically distinct. Shri, what problems might this cause? I don’t understand the figure you’ve drawn above. Sorry. As a user, would I need to add anything to the way I am setting up the network/plex or any additional equations in the residual evaluation? I do not have any issue right now for the power grid problem since I don’t require DMNetwork or DMPLEX to do the topological distinction between parallel edges. There are unique edge identifiers in my dataset through which I can make this distinction. Yes, this would work, but it looks like the multiple cells are not causing them problems right now with the questions they are asking the mesh. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/<https://www.google.com/url?q=https://gcc02.safelinks.protection.outlook.com…> <DiestelGT.pdf>

1 0

Re: [petsc-dev] mpi_comm_create_group()
by Jed Brown 02 Dec '21

02 Dec '21

Adrian Croucher <a.croucher(a)auckland.ac.nz> writes: > hi > > This question is not strictly PETSc-related but I figure you guys are > probably most likely to know the answer. > > I need to do some simple computations involving only a small subset of > the parallel processes in my simulation. > > From reading the MPI documentation and a few tutorials it looks like > mpi_comm_create_group() is probably the thing to use for creating the > MPI communicator in this case. Since there are only a few processes in > the group, from what I can gather this function should be more efficient > than mpi_comm_create(), as it's only collective on the group, not the > whole parent communicator (MPI_COMM_WORLD). > > 1) Is it correct that mpi_comm_create_group() should be a better option > in this case than mpi_comm_create()? I had a grep through the PETSc > source code and there are some calls to mpi_comm_create() but none to > mpi_comm_create_group(). But maybe the use case is different. Sure, but this is probably premature optimization. Many find MPI_Comm_split() to be a more convenient interface. > 2) If mpi_comm_create_group() is the better option, is it necessary to > call it on all processes, or only the ones in the group? The tutorial at > https://mpitutorial.com/tutorials/introduction-to-groups-and-communicators/ > calls it on all processes, but other stuff I've read suggests you only > need to call it on processes in the group. It seems to work either way, > but you have to use the communicator a little differently. I think the man page is fairly clear. | MPI_Comm_create_group is similar to MPI_Comm_create; however, MPI_Comm_create must be called by all processes in the group of comm, whereas MPI_Comm_create_group must be called by all processes in group, which is a subgroup of the group of comm.

1 0

Re: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot?
by Barry Smith 02 Dec '21

02 Dec '21

Aiden, Do you have true degrees of freedom on both the edges and the vertices? If only have edge variables and no vertex variables but only use "work" values on vertices than maybe DMNETWORK might behave as you need. But my trivial example defintely shows a problem with only edge vertex variables. The DMPLEX object believes there are more than two vertices when there are only two because it cannot determine that some vertices are actually the same. Barry > On Dec 2, 2021, at 5:31 PM, Zhang, Hong via petsc-dev <petsc-dev(a)mcs.anl.gov> wrote: > > All, > I'm forwarding Aidan's comments on this issue (see below). Aidan has done extensive study on DMPlex and has been working on hyperbolic flow applications. > > Satish: > Please add Aidan to petsc-dev mailing list. > > Hong > From: Aidan Hamilton <aidan(a)udel.edu <mailto:[email protected]>> > Sent: Thursday, December 2, 2021 2:34 PM > To: Zhang, Hong <hzhang(a)mcs.anl.gov <mailto:[email protected]>> > Cc: Sundermann, Rylee Edwin - SDSU Student <rylee.sundermann(a)jacks.sdstate.edu <mailto:[email protected]>> > Subject: Re: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot? > > Hong, > > I am not on petsc-dev, I guess I should be. Also you can push, you’re always free to push changes. > > And it doesn’t matter for anything on my end. Each edge is distinct topological entity, so I have the directed graph > > V0 ---- E0 - V1 ----- E1 ---- V0 > > I don’t need any special identification. And it works in parallel with globaltolocal seemingly working . Now this is not a standard graph, it follows the more general notion of directed graph, see Diestel page 25, that allows for multi-edges. > > Now for my case I don’t need to distinguish between the two edges at all beyond each one having their own unique edge id. In all of our use cases for traffic I don’t think we need to care (and honestly I’m not sure if Shri actually needs to care either). > > Now we do have the topological distinction of direction for all of our graphs, which we can use to assign direction to traffic flow. And that will always work. And this notion of direction is the only distinction we need to care about, and we have it. > > It’s worth noting that a CW-Complex does support this notion of multiedges, no reason for it not to, so DMPlex should be supporting its topological operations with this case in mind. > If DMplex can’t support this then dmplex isn’t doing what it claims it does. > > As long as you as the user know that you are dealing with a multi-edge graph, I don’t see why there would be any issues. Any topological query of a multi-edge graph should behave as you would expect a multi-edge graph to behave, though it may behave differently than single edge graphs, as long as you are aware of the difference when you code your residuals and other functions on dmnetwork it should be fine. > > Best, > > Aidan > > > From: Zhang, Hong <hzhang(a)mcs.anl.gov <mailto:[email protected]>> > Date: Thursday, December 2, 2021 at 1:56 PM > To: Aidan Hamilton <aidan(a)udel.edu <mailto:[email protected]>> > Cc: Sundermann, Rylee Edwin - SDSU Student <rylee.sundermann(a)jacks.sdstate.edu <mailto:[email protected]>> > Subject: Fw: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot? > > Aidan, > Are you on petsc-dev email list? > How do you distinguish two parallel edges connecting the same junctions in your traffic code? Do you use > offset_vto, offset_vfrom in EdgeFE? > > I removed white spaces in dgnet.c and dgnet.h. Can I push the changes? > Hong > From: petsc-dev <petsc-dev-bounces(a)mcs.anl.gov <mailto:[email protected]>> on behalf of Barry Smith <bsmith(a)petsc.dev <mailto:[email protected]>> > Sent: Wednesday, December 1, 2021 7:41 PM > To: Abhyankar, Shrirang G <shrirang.abhyankar(a)pnnl.gov <mailto:[email protected]>> > Cc: petsc-dev <petsc-dev(a)mcs.anl.gov <mailto:[email protected]>>; Betrie, Getnet <gbetrie(a)anl.gov <mailto:[email protected]>> > Subject: Re: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot? > > > Shri, > > If you provided your graph in parallel I suspect that the GlobalToLocal and LocalToGlobal would not work as you expect and put everything in the appropriate place. > > I'll set up a simple code that takes exactly Get's configuration and send it out so everyone can run it and see exactly what happens. > > Barry > > > > On Dec 1, 2021, at 8:00 PM, Abhyankar, Shrirang G via petsc-dev <petsc-dev(a)mcs.anl.gov <mailto:[email protected]>> wrote: > > > > From: Matthew Knepley <knepley(a)gmail.com <mailto:[email protected]>> > Date: Wednesday, December 1, 2021 at 5:57 PM > To: "Brown, Jedediah A (VISIT)" <jed(a)jedbrown.org <mailto:[email protected]>> > Cc: "Abhyankar, Shrirang G" <shrirang.abhyankar(a)pnnl.gov <mailto:[email protected]>>, PETSc Development <petsc-dev(a)mcs.anl.gov <mailto:[email protected]>>, Getnet Betrie <gbetrie(a)anl.gov <mailto:[email protected]>> > Subject: Re: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot? > > Check twice before you click! This email originated from outside PNNL. > > On Wed, Dec 1, 2021 at 6:55 PM Jed Brown <jed(a)jedbrown.org <mailto:[email protected]>> wrote: > Matthew Knepley <knepley(a)gmail.com <mailto:[email protected]>> writes: > > > On Wed, Dec 1, 2021 at 5:17 PM Abhyankar, Shrirang G < > > shrirang.abhyankar(a)pnnl.gov <mailto:[email protected]>> wrote: > > > >> “You can certainly have many fields on a given edge, but I don't know > >> what it would mean to have two edges since no topological query could tell > >> the difference.” > >> > >> > >> > >> The two edges in a power grid represent two parallel power lines that are > >> connected between two locations (vertices). There are line ids (stored in > >> the component data) to distinguish the two lines. > >> > > > > Yes, so you can tell the difference in the function space (since difference > > current passes down each one), but _topologically_ you cannot. If you put > > duplicate cells in, then > > some topological queries will give unexpected results, like the join of the > > two vertices. > > This could be modeled with some ghost vertices. So instead of > > a ------ b > \_____/ > > you would set up > > a ---o---- b > \___o___/ > > Those ghost vertices don't have to "do" anything, but they make the edges topologically distinct. > > Shri, what problems might this cause? > > I don’t understand the figure you’ve drawn above. Sorry. > > As a user, would I need to add anything to the way I am setting up the network/plex or any additional equations in the residual evaluation? > > I do not have any issue right now for the power grid problem since I don’t require DMNetwork or DMPLEX to do the topological distinction between parallel edges. There are unique edge identifiers in my dataset through which I can make this distinction. > > > Yes, this would work, but it looks like the multiple cells are not causing them problems right now with the questions they are asking the mesh. > > Matt > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ <https://www.google.com/url?q=https://gcc02.safelinks.protection.outlook.com…> > > <DiestelGT.pdf>

1 0

mpi_comm_create_group()
by Adrian Croucher 02 Dec '21

02 Dec '21

hi This question is not strictly PETSc-related but I figure you guys are probably most likely to know the answer. I need to do some simple computations involving only a small subset of the parallel processes in my simulation. From reading the MPI documentation and a few tutorials it looks like mpi_comm_create_group() is probably the thing to use for creating the MPI communicator in this case. Since there are only a few processes in the group, from what I can gather this function should be more efficient than mpi_comm_create(), as it's only collective on the group, not the whole parent communicator (MPI_COMM_WORLD). 1) Is it correct that mpi_comm_create_group() should be a better option in this case than mpi_comm_create()? I had a grep through the PETSc source code and there are some calls to mpi_comm_create() but none to mpi_comm_create_group(). But maybe the use case is different. 2) If mpi_comm_create_group() is the better option, is it necessary to call it on all processes, or only the ones in the group? The tutorial at https://mpitutorial.com/tutorials/introduction-to-groups-and-communicators/ calls it on all processes, but other stuff I've read suggests you only need to call it on processes in the group. It seems to work either way, but you have to use the communicator a little differently. - Adrian -- Dr Adrian Croucher Senior Research Fellow Department of Engineering Science University of Auckland, New Zealand email: a.croucher(a)auckland.ac.nz tel: +64 (0)9 923 4611

1 0

Fw: DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot?
by Zhang, Hong 02 Dec '21

02 Dec '21

All, I'm forwarding Aidan's comments on this issue (see below). Aidan has done extensive study on DMPlex and has been working on hyperbolic flow applications. Satish: Please add Aidan to petsc-dev mailing list. Hong ________________________________ From: Aidan Hamilton <aidan(a)udel.edu> Sent: Thursday, December 2, 2021 2:34 PM To: Zhang, Hong <hzhang(a)mcs.anl.gov> Cc: Sundermann, Rylee Edwin - SDSU Student <rylee.sundermann(a)jacks.sdstate.edu> Subject: Re: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot? Hong, I am not on petsc-dev, I guess I should be. Also you can push, you’re always free to push changes. And it doesn’t matter for anything on my end. Each edge is distinct topological entity, so I have the directed graph V0 ---- E0 -• V1 ----- E1 ----• V0 I don’t need any special identification. And it works in parallel with globaltolocal seemingly working . Now this is not a standard graph, it follows the more general notion of directed graph, see Diestel page 25, that allows for multi-edges. Now for my case I don’t need to distinguish between the two edges at all beyond each one having their own unique edge id. In all of our use cases for traffic I don’t think we need to care (and honestly I’m not sure if Shri actually needs to care either). Now we do have the topological distinction of direction for all of our graphs, which we can use to assign direction to traffic flow. And that will always work. And this notion of direction is the only distinction we need to care about, and we have it. It’s worth noting that a CW-Complex does support this notion of multiedges, no reason for it not to, so DMPlex should be supporting its topological operations with this case in mind. If DMplex can’t support this then dmplex isn’t doing what it claims it does. As long as you as the user know that you are dealing with a multi-edge graph, I don’t see why there would be any issues. Any topological query of a multi-edge graph should behave as you would expect a multi-edge graph to behave, though it may behave differently than single edge graphs, as long as you are aware of the difference when you code your residuals and other functions on dmnetwork it should be fine. Best, Aidan From: Zhang, Hong <hzhang(a)mcs.anl.gov> Date: Thursday, December 2, 2021 at 1:56 PM To: Aidan Hamilton <aidan(a)udel.edu> Cc: Sundermann, Rylee Edwin - SDSU Student <rylee.sundermann(a)jacks.sdstate.edu> Subject: Fw: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot? Aidan, Are you on petsc-dev email list? How do you distinguish two parallel edges connecting the same junctions in your traffic code? Do you use offset_vto, offset_vfrom in EdgeFE? I removed white spaces in dgnet.c and dgnet.h. Can I push the changes? Hong ________________________________ From: petsc-dev <petsc-dev-bounces(a)mcs.anl.gov> on behalf of Barry Smith <bsmith(a)petsc.dev> Sent: Wednesday, December 1, 2021 7:41 PM To: Abhyankar, Shrirang G <shrirang.abhyankar(a)pnnl.gov> Cc: petsc-dev <petsc-dev(a)mcs.anl.gov>; Betrie, Getnet <gbetrie(a)anl.gov> Subject: Re: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot? Shri, If you provided your graph in parallel I suspect that the GlobalToLocal and LocalToGlobal would not work as you expect and put everything in the appropriate place. I'll set up a simple code that takes exactly Get's configuration and send it out so everyone can run it and see exactly what happens. Barry On Dec 1, 2021, at 8:00 PM, Abhyankar, Shrirang G via petsc-dev <petsc-dev(a)mcs.anl.gov<mailto:[email protected]>> wrote: From: Matthew Knepley <knepley(a)gmail.com<mailto:[email protected]>> Date: Wednesday, December 1, 2021 at 5:57 PM To: "Brown, Jedediah A (VISIT)" <jed(a)jedbrown.org<mailto:[email protected]>> Cc: "Abhyankar, Shrirang G" <shrirang.abhyankar(a)pnnl.gov<mailto:[email protected]>>, PETSc Development <petsc-dev(a)mcs.anl.gov<mailto:[email protected]>>, Getnet Betrie <gbetrie(a)anl.gov<mailto:[email protected]>> Subject: Re: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot? Check twice before you click! This email originated from outside PNNL. On Wed, Dec 1, 2021 at 6:55 PM Jed Brown <jed(a)jedbrown.org<mailto:[email protected]>> wrote: Matthew Knepley <knepley(a)gmail.com<mailto:[email protected]>> writes: > On Wed, Dec 1, 2021 at 5:17 PM Abhyankar, Shrirang G < > shrirang.abhyankar(a)pnnl.gov<mailto:[email protected]>> wrote: > >> “You can certainly have many fields on a given edge, but I don't know >> what it would mean to have two edges since no topological query could tell >> the difference.” >> >> >> >> The two edges in a power grid represent two parallel power lines that are >> connected between two locations (vertices). There are line ids (stored in >> the component data) to distinguish the two lines. >> > > Yes, so you can tell the difference in the function space (since difference > current passes down each one), but _topologically_ you cannot. If you put > duplicate cells in, then > some topological queries will give unexpected results, like the join of the > two vertices. This could be modeled with some ghost vertices. So instead of a ------ b \_____/ you would set up a ---o---- b \___o___/ Those ghost vertices don't have to "do" anything, but they make the edges topologically distinct. Shri, what problems might this cause? I don’t understand the figure you’ve drawn above. Sorry. As a user, would I need to add anything to the way I am setting up the network/plex or any additional equations in the residual evaluation? I do not have any issue right now for the power grid problem since I don’t require DMNetwork or DMPLEX to do the topological distinction between parallel edges. There are unique edge identifiers in my dataset through which I can make this distinction. Yes, this would work, but it looks like the multiple cells are not causing them problems right now with the questions they are asking the mesh. Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/<https://www.google.com/url?q=https://gcc02.safelinks.protection.outlook.com…>

1 0

Re: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot?
by Barry Smith 02 Dec '21

02 Dec '21

> On Dec 2, 2021, at 12:55 PM, Matthew Knepley <knepley(a)gmail.com> wrote: > > On Thu, Dec 2, 2021 at 12:07 PM Barry Smith <bsmith(a)petsc.dev <mailto:[email protected]>> wrote: > > I have pushed a branch https://gitlab.com/petsc/petsc/-/merge_requests/4612 <https://gitlab.com/petsc/petsc/-/merge_requests/4612> that generates the same edge on two ranks and demonstrates the totally inappropriate PetscSF and vectors that are generated in this case (just look at the output file in the MR changes) > > I will go through this carefully. However, I want to note that you are looking at the Section SF, not the Point SF. Correct. > How many variables > are there in your problem? In this case I put one variable on each vertex and no variables on the edges (same model as for Neuron's). > Also, can you print the pointSF as well? This can be trivially added to the example. > > Thanks, > > Matt > > Thoughts on what we should do? I think Jed's idea of dummy vertices is very cumbersome and ugly unless it can be refined somehow. Is there a different way DMPLEX could be utilized to easily support multiple edges? Maybe not using DMPlexBuildFromCellListParallel(), but building the plex some other way? Note that it is clearly theoretically possible to have DMPlexBuildFromCellListParallel() detect a problem and error but I don't know how to do it. > > Barry > > > > >> On Dec 1, 2021, at 8:00 PM, Abhyankar, Shrirang G via petsc-dev <petsc-dev(a)mcs.anl.gov <mailto:[email protected]>> wrote: >> >> >> >> From: Matthew Knepley <knepley(a)gmail.com <mailto:[email protected]>> >> Date: Wednesday, December 1, 2021 at 5:57 PM >> To: "Brown, Jedediah A (VISIT)" <jed(a)jedbrown.org <mailto:[email protected]>> >> Cc: "Abhyankar, Shrirang G" <shrirang.abhyankar(a)pnnl.gov <mailto:[email protected]>>, PETSc Development <petsc-dev(a)mcs.anl.gov <mailto:[email protected]>>, Getnet Betrie <gbetrie(a)anl.gov <mailto:[email protected]>> >> Subject: Re: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot? >> >> Check twice before you click! This email originated from outside PNNL. >> >> On Wed, Dec 1, 2021 at 6:55 PM Jed Brown <jed(a)jedbrown.org <mailto:[email protected]>> wrote: >>> Matthew Knepley <knepley(a)gmail.com <mailto:[email protected]>> writes: >>> >>> > On Wed, Dec 1, 2021 at 5:17 PM Abhyankar, Shrirang G < >>> > shrirang.abhyankar(a)pnnl.gov <mailto:[email protected]>> wrote: >>> > >>> >> “You can certainly have many fields on a given edge, but I don't know >>> >> what it would mean to have two edges since no topological query could tell >>> >> the difference.” >>> >> >>> >> >>> >> >>> >> The two edges in a power grid represent two parallel power lines that are >>> >> connected between two locations (vertices). There are line ids (stored in >>> >> the component data) to distinguish the two lines. >>> >> >>> > >>> > Yes, so you can tell the difference in the function space (since difference >>> > current passes down each one), but _topologically_ you cannot. If you put >>> > duplicate cells in, then >>> > some topological queries will give unexpected results, like the join of the >>> > two vertices. >>> >>> This could be modeled with some ghost vertices. So instead of >>> >>> a ------ b >>> \_____/ >>> >>> you would set up >>> >>> a ---o---- b >>> \___o___/ >>> >>> Those ghost vertices don't have to "do" anything, but they make the edges topologically distinct. >>> >>> Shri, what problems might this cause? >> >> >> I don’t understand the figure you’ve drawn above. Sorry. >> >> As a user, would I need to add anything to the way I am setting up the network/plex or any additional equations in the residual evaluation? >> >> I do not have any issue right now for the power grid problem since I don’t require DMNetwork or DMPLEX to do the topological distinction between parallel edges. There are unique edge identifiers in my dataset through which I can make this distinction. >> >> >> Yes, this would work, but it looks like the multiple cells are not causing them problems right now with the questions they are asking the mesh. >> >> Matt >> >> -- >> What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ <https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cse.bu…> > > > -- > What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

1 0

Re: [petsc-dev] DMPLEX cannot support two different edges for the same two vertices, hence DMPLEX cannot?
by Matthew Knepley 02 Dec '21

02 Dec '21

On Thu, Dec 2, 2021 at 12:07 PM Barry Smith <bsmith(a)petsc.dev> wrote: > > I have pushed a branch > https://gitlab.com/petsc/petsc/-/merge_requests/4612 that generates the > same edge on two ranks and demonstrates the totally inappropriate PetscSF > and vectors that are generated in this case (just look at the output file > in the MR changes) > I will go through this carefully. However, I want to note that you are looking at the Section SF, not the Point SF. How many variables are there in your problem? Also, can you print the pointSF as well? Thanks, Matt > Thoughts on what we should do? I think Jed's idea of dummy vertices is > very cumbersome and ugly unless it can be refined somehow. Is there a > different way DMPLEX could be utilized to easily support multiple edges? > Maybe not using DMPlexBuildFromCellListParallel(), but building the plex > some other way? Note that it is clearly theoretically possible to > have DMPlexBuildFromCellListParallel() detect a problem and error but I > don't know how to do it. > > Barry > > > > > On Dec 1, 2021, at 8:00 PM, Abhyankar, Shrirang G via petsc-dev < > petsc-dev(a)mcs.anl.gov> wrote: > > > > *From: *Matthew Knepley <knepley(a)gmail.com> > *Date: *Wednesday, December 1, 2021 at 5:57 PM > *To: *"Brown, Jedediah A (VISIT)" <jed(a)jedbrown.org> > *Cc: *"Abhyankar, Shrirang G" <shrirang.abhyankar(a)pnnl.gov>, PETSc > Development <petsc-dev(a)mcs.anl.gov>, Getnet Betrie <gbetrie(a)anl.gov> > *Subject: *Re: [petsc-dev] DMPLEX cannot support two different edges for > the same two vertices, hence DMPLEX cannot? > > Check twice before you click! This email originated from outside PNNL. > > On Wed, Dec 1, 2021 at 6:55 PM Jed Brown <jed(a)jedbrown.org> wrote: > > Matthew Knepley <knepley(a)gmail.com> writes: > > > On Wed, Dec 1, 2021 at 5:17 PM Abhyankar, Shrirang G < > > shrirang.abhyankar(a)pnnl.gov> wrote: > > > >> “You can certainly have many fields on a given edge, but I don't know > >> what it would mean to have two edges since no topological query could > tell > >> the difference.” > >> > >> > >> > >> The two edges in a power grid represent two parallel power lines that > are > >> connected between two locations (vertices). There are line ids (stored > in > >> the component data) to distinguish the two lines. > >> > > > > Yes, so you can tell the difference in the function space (since > difference > > current passes down each one), but _topologically_ you cannot. If you put > > duplicate cells in, then > > some topological queries will give unexpected results, like the join of > the > > two vertices. > > This could be modeled with some ghost vertices. So instead of > > a ------ b > \_____/ > > you would set up > > a ---o---- b > \___o___/ > > Those ghost vertices don't have to "do" anything, but they make the edges > topologically distinct. > > Shri, what problems might this cause? > > > I don’t understand the figure you’ve drawn above. Sorry. > > As a user, would I need to add anything to the way I am setting up the > network/plex or any additional equations in the residual evaluation? > > I do not have any issue right now for the power grid problem since I don’t > require DMNetwork or DMPLEX to do the topological distinction between > parallel edges. There are unique edge identifiers in my dataset through > which I can make this distinction. > > > Yes, this would work, but it looks like the multiple cells are not causing > them problems right now with the questions they are asking the mesh. > > Matt > > -- > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > https://www.cse.buffalo.edu/~knepley/ > <https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cse.bu…> > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

1 0