Union of sequential vecs
Hi, I want to take the union of a set of sequential vectors, each living in a different processor. Say, Vec_Seq1 = {2,5,7} Vec_Seq2 = {5,8,10,11} Vec_Seq3 = {5,2,12}. Finally, get the union of all them Vec = {2,5,7,8,10,11,12}. I initially wanted to create a parallel vector and insert the (sequential vector) values but I do not know, to which index to insert the values to. But I do know the total size of Vec (which in this case is 7). Any help is much appreciated. Kind regards, Karthik. This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
How are you combining them to get Vec = {2,5,7,8,10,11,12}? Do you want the values to remain on the same MPI rank as before, just in an MPI vector?
On Dec 9, 2022, at 2:28 PM, Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected]> wrote:
Hi,
I want to take the union of a set of sequential vectors, each living in a different processor.
Say, Vec_Seq1 = {2,5,7} Vec_Seq2 = {5,8,10,11} Vec_Seq3 = {5,2,12}.
Finally, get the union of all them Vec = {2,5,7,8,10,11,12}.
I initially wanted to create a parallel vector and insert the (sequential vector) values but I do not know, to which index to insert the values to. But I do know the total size of Vec (which in this case is 7).
Any help is much appreciated.
Kind regards, Karthik.
This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
That is where I am stuck, I don’t know who to combine them to get Vec = {2,5,7,8,10,11,12}. I just want them in an MPI vector. I finally plan to call VecScatterCreateToAll so that all processor gets a copy. Thank you. Kind regards, Karthik. From: Barry Smith <[email protected]> Date: Friday, 9 December 2022 at 20:04 To: Chockalingam, Karthikeyan (STFC,DL,HC) <[email protected]> Cc: [email protected] <[email protected]> Subject: Re: [petsc-users] Union of sequential vecs How are you combining them to get Vec = {2,5,7,8,10,11,12}? Do you want the values to remain on the same MPI rank as before, just in an MPI vector? On Dec 9, 2022, at 2:28 PM, Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected]> wrote: Hi, I want to take the union of a set of sequential vectors, each living in a different processor. Say, Vec_Seq1 = {2,5,7} Vec_Seq2 = {5,8,10,11} Vec_Seq3 = {5,2,12}. Finally, get the union of all them Vec = {2,5,7,8,10,11,12}. I initially wanted to create a parallel vector and insert the (sequential vector) values but I do not know, to which index to insert the values to. But I do know the total size of Vec (which in this case is 7). Any help is much appreciated. Kind regards, Karthik. This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
If your space is pretty compact, eg, (0,12), you could create an MPI vector Q of size 13, say, and each processor can add 1.0 to Q[Vec[i]], for all "i" in my local "Vec". Then each processor can count the number of local nonzeros in Q, call it n, create a new vector, R, with local size n, then set R[i] = global index of the nonzero for each nonzero in Q, i=0:n. Do some sort of vec-scatter-to-all with R to get what you want. Does that work? Mark On Fri, Dec 9, 2022 at 3:25 PM Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected]> wrote:
That is where I am stuck, *I don’t know* who to combine them to get Vec = {2,5,7,8,10,11,12}.
I just want them in an MPI vector.
I finally plan to call VecScatterCreateToAll so that all processor gets a copy.
Thank you.
Kind regards,
Karthik.
*From: *Barry Smith <[email protected]> *Date: *Friday, 9 December 2022 at 20:04 *To: *Chockalingam, Karthikeyan (STFC,DL,HC) < [email protected]> *Cc: *[email protected] <[email protected]> *Subject: *Re: [petsc-users] Union of sequential vecs
How are you combining them to get Vec = {2,5,7,8,10,11,12}?
Do you want the values to remain on the same MPI rank as before, just in an MPI vector?
On Dec 9, 2022, at 2:28 PM, Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected]> wrote:
Hi,
I want to take the union of a set of sequential vectors, each living in a different processor.
Say,
Vec_Seq1 = {2,5,7}
Vec_Seq2 = {5,8,10,11}
Vec_Seq3 = {5,2,12}.
Finally, get the union of all them Vec = {2,5,7,8,10,11,12}.
I initially wanted to create a parallel vector and insert the (sequential vector) values but I do not know, to which index to insert the values to. But I do know the total size of Vec (which in this case is 7).
Any help is much appreciated.
Kind regards,
Karthik.
This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
Thank you Mark and Barry. @Mark Adams<mailto:[email protected]> I follow you for the most part. Shouldn’t R be an MPI Vector? Here it goes: Q[Vec[i]] for all "i" in my local (sequential) "Vec". Compute number of local nonzeros in Q, call it n. //Create a MPI Vector R, with local size n VecCreate(PETSC_COMM_WORLD, &R); VecSetType(R, VECMPI); VecSetSizes(R, n, PETSC_DECIDE); //Populate MPI Vector R with local (sequential) "Vec". VecGetOwnershipRange(R, &istart, &iend); local_size = iend - istart; //The local_size should be ‘n’ right? VecGetArray(R, &values); for (i = 0; i < local_size; i++) { values[i] = Vec[i]; } VecRestoreArray(R, &values); //Scatter R to all processors Vec V_SEQ; VecScatter ctx; VecScatterCreateToAll(R,&ctx,&V_SEQ); //Remove duplicates in V_SEQ How can I use PetscSortRemoveDupsInt to remove duplicates in V_SEQ? Physics behind: I am reading a parallel mesh, and want to mark all the boundary nodes. I use a local (sequential) Vec to store the boundary nodes for each parallel partition. Hence, local Vecs can end up with duplicate node index among them, which I would like to get rid of when I combine all of them together. Best, Karthik. From: Mark Adams <[email protected]> Date: Friday, 9 December 2022 at 21:08 To: Chockalingam, Karthikeyan (STFC,DL,HC) <[email protected]> Cc: Barry Smith <[email protected]>, [email protected] <[email protected]> Subject: Re: [petsc-users] Union of sequential vecs If your space is pretty compact, eg, (0,12), you could create an MPI vector Q of size 13, say, and each processor can add 1.0 to Q[Vec[i]], for all "i" in my local "Vec". Then each processor can count the number of local nonzeros in Q, call it n, create a new vector, R, with local size n, then set R[i] = global index of the nonzero for each nonzero in Q, i=0:n. Do some sort of vec-scatter-to-all with R to get what you want. Does that work? Mark On Fri, Dec 9, 2022 at 3:25 PM Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected]<mailto:[email protected]>> wrote: That is where I am stuck, I don’t know who to combine them to get Vec = {2,5,7,8,10,11,12}. I just want them in an MPI vector. I finally plan to call VecScatterCreateToAll so that all processor gets a copy. Thank you. Kind regards, Karthik. From: Barry Smith <[email protected]<mailto:[email protected]>> Date: Friday, 9 December 2022 at 20:04 To: Chockalingam, Karthikeyan (STFC,DL,HC) <[email protected]<mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Subject: Re: [petsc-users] Union of sequential vecs How are you combining them to get Vec = {2,5,7,8,10,11,12}? Do you want the values to remain on the same MPI rank as before, just in an MPI vector? On Dec 9, 2022, at 2:28 PM, Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected]<mailto:[email protected]>> wrote: Hi, I want to take the union of a set of sequential vectors, each living in a different processor. Say, Vec_Seq1 = {2,5,7} Vec_Seq2 = {5,8,10,11} Vec_Seq3 = {5,2,12}. Finally, get the union of all them Vec = {2,5,7,8,10,11,12}. I initially wanted to create a parallel vector and insert the (sequential vector) values but I do not know, to which index to insert the values to. But I do know the total size of Vec (which in this case is 7). Any help is much appreciated. Kind regards, Karthik. This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
On Fri, Dec 9, 2022 at 6:50 PM Karthikeyan Chockalingam - STFC UKRI < [email protected]> wrote:
Thank you Mark and Barry.
@Mark Adams <[email protected]> I follow you for the most part. Shouldn’t R be an MPI Vector?
yes, "R, with local size n" implies an MPI vector.
Here it goes:
Q[Vec[i]] for all "i" in my local (sequential) "Vec".
OK, you are calling VecSetValue(Q,... ADD_VALUES <https://petsc.org/main/docs/manualpages/Sys/ADD_VALUES/>) VecAsseemblyBegin/End needs to be called and that will do communication.
Compute number of local nonzeros in Q, call it n.
You would use VecGetArray like below to do this count. You could cache the indices in Q that are non-zero, or redo the loop below.
//Create a MPI Vector R, with local size n
VecCreate(PETSC_COMM_WORLD, &R);
VecSetType(R, VECMPI);
VecSetSizes(R, *n*, PETSC_DECIDE);
//Populate MPI Vector R with local (sequential) "Vec".
VecGetOwnershipRange(R, &istart, &iend);
local_size = iend - istart; //The local_size should be ‘*n*’ right?
Well yes, but you are going to need to get the "istart" from Q, not R, to get the global index in a loop.
VecGetArray(R, &values);
No, you want to get the values from Q and use VecSetValue(R,... VecGetArray(Q, &values_Q); VecGetOwnershipRange(R, &istart_R, NULL); idx = istart_R
*for* (i = 0; i < local_size_Q; i++) {
No, redo the loop above that you used to count. You skipped this so I can put it here. (simpler to just run this loop twice, first count and then set) If (values_Q[i] != 0) [ VecSetValue(R, idx++, (PetscScalar) istart_Q+i, INSERT_VALUES <https://petsc.org/main/docs/manualpages/Sys/INSERT_VALUES/>) }
values[i] = Vec[i];
}
VecRestoreArray(R, &values);
VecRestoreArray(Q, &values_Q); You don't really need a VecAsseemblyBegin/End (R) here but you can add it to be clear. I'm not sure this is correct so you need to debug this and the scatter can be figured out later.
//Scatter R to all processors
Vec V_SEQ;
VecScatter ctx;
VecScatterCreateToAll(R,&ctx,&V_SEQ);
I'm not sure how to best do this. Look in the docs or ask in a separate thread. This thread is busy figuring the first part out.
//Remove duplicates in V_SEQ
How can I use PetscSortRemoveDupsInt to remove duplicates in V_SEQ?
Physics behind:
I am reading a parallel mesh, and want to mark all the boundary nodes. I use a local (sequential) Vec to store the boundary nodes for each parallel partition. Hence, local Vecs can end up with duplicate node index among them, which I would like to get rid of when I combine all of them together.
Humm, OK, I'm not sure I get this exactly but yes this is intended to get each process a global list of (boundary) vertices. Not super scalable, but if it gets you started then that's great. Good luck, Mark
Best,
Karthik.
*From: *Mark Adams <[email protected]> *Date: *Friday, 9 December 2022 at 21:08 *To: *Chockalingam, Karthikeyan (STFC,DL,HC) < [email protected]> *Cc: *Barry Smith <[email protected]>, [email protected] < [email protected]> *Subject: *Re: [petsc-users] Union of sequential vecs
If your space is pretty compact, eg, (0,12), you could create an MPI vector Q of size 13, say, and each processor can add 1.0 to Q[Vec[i]], for all "i" in my local "Vec".
Then each processor can count the number of local nonzeros in Q, call it n, create a new vector, R, with local size n, then set R[i] = global index of the nonzero for each nonzero in Q, i=0:n.
Do some sort of vec-scatter-to-all with R to get what you want.
Does that work?
Mark
On Fri, Dec 9, 2022 at 3:25 PM Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected]> wrote:
That is where I am stuck, *I don’t know* who to combine them to get Vec = {2,5,7,8,10,11,12}.
I just want them in an MPI vector.
I finally plan to call VecScatterCreateToAll so that all processor gets a copy.
Thank you.
Kind regards,
Karthik.
*From: *Barry Smith <[email protected]> *Date: *Friday, 9 December 2022 at 20:04 *To: *Chockalingam, Karthikeyan (STFC,DL,HC) < [email protected]> *Cc: *[email protected] <[email protected]> *Subject: *Re: [petsc-users] Union of sequential vecs
How are you combining them to get Vec = {2,5,7,8,10,11,12}?
Do you want the values to remain on the same MPI rank as before, just in an MPI vector?
On Dec 9, 2022, at 2:28 PM, Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected]> wrote:
Hi,
I want to take the union of a set of sequential vectors, each living in a different processor.
Say,
Vec_Seq1 = {2,5,7}
Vec_Seq2 = {5,8,10,11}
Vec_Seq3 = {5,2,12}.
Finally, get the union of all them Vec = {2,5,7,8,10,11,12}.
I initially wanted to create a parallel vector and insert the (sequential vector) values but I do not know, to which index to insert the values to. But I do know the total size of Vec (which in this case is 7).
Any help is much appreciated.
Kind regards,
Karthik.
This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
On Fri, Dec 9, 2022 at 6:50 PM Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected]> wrote:
Thank you Mark and Barry.
@Mark Adams <[email protected]> I follow you for the most part. Shouldn’t R be an MPI Vector?
Here it goes:
Q[Vec[i]] for all "i" in my local (sequential) "Vec".
Compute number of local nonzeros in Q, call it n.
//Create a MPI Vector R, with local size n
VecCreate(PETSC_COMM_WORLD, &R);
VecSetType(R, VECMPI);
VecSetSizes(R, *n*, PETSC_DECIDE);
//Populate MPI Vector R with local (sequential) "Vec".
VecGetOwnershipRange(R, &istart, &iend);
local_size = iend - istart; //The local_size should be ‘*n*’ right?
VecGetArray(R, &values);
*for* (i = 0; i < local_size; i++) {
values[i] = Vec[i];
}
VecRestoreArray(R, &values);
//Scatter R to all processors
Vec V_SEQ;
VecScatter ctx;
VecScatterCreateToAll(R,&ctx,&V_SEQ);
//Remove duplicates in V_SEQ
How can I use PetscSortRemoveDupsInt to remove duplicates in V_SEQ?
Physics behind:
I am reading a parallel mesh, and want to mark all the boundary nodes. I use a local (sequential) Vec to store the boundary nodes for each parallel partition. Hence, local Vecs can end up with duplicate node index among them, which I would like to get rid of when I combine all of them together.
1) Blaise is right you should use an IS, not a Vec, to hold node indices. His solution is only a few lines, so I would use it. 2) I would not recommend doing things this way in the first place. PETSc can manage parallel meshes scalably, marking boundaries using DMLabel objects. Thanks, Matt
Best,
Karthik.
*From: *Mark Adams <[email protected]> *Date: *Friday, 9 December 2022 at 21:08 *To: *Chockalingam, Karthikeyan (STFC,DL,HC) < [email protected]> *Cc: *Barry Smith <[email protected]>, [email protected] < [email protected]> *Subject: *Re: [petsc-users] Union of sequential vecs
If your space is pretty compact, eg, (0,12), you could create an MPI vector Q of size 13, say, and each processor can add 1.0 to Q[Vec[i]], for all "i" in my local "Vec".
Then each processor can count the number of local nonzeros in Q, call it n, create a new vector, R, with local size n, then set R[i] = global index of the nonzero for each nonzero in Q, i=0:n.
Do some sort of vec-scatter-to-all with R to get what you want.
Does that work?
Mark
On Fri, Dec 9, 2022 at 3:25 PM Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected]> wrote:
That is where I am stuck, *I don’t know* who to combine them to get Vec = {2,5,7,8,10,11,12}.
I just want them in an MPI vector.
I finally plan to call VecScatterCreateToAll so that all processor gets a copy.
Thank you.
Kind regards,
Karthik.
*From: *Barry Smith <[email protected]> *Date: *Friday, 9 December 2022 at 20:04 *To: *Chockalingam, Karthikeyan (STFC,DL,HC) < [email protected]> *Cc: *[email protected] <[email protected]> *Subject: *Re: [petsc-users] Union of sequential vecs
How are you combining them to get Vec = {2,5,7,8,10,11,12}?
Do you want the values to remain on the same MPI rank as before, just in an MPI vector?
On Dec 9, 2022, at 2:28 PM, Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected]> wrote:
Hi,
I want to take the union of a set of sequential vectors, each living in a different processor.
Say,
Vec_Seq1 = {2,5,7}
Vec_Seq2 = {5,8,10,11}
Vec_Seq3 = {5,2,12}.
Finally, get the union of all them Vec = {2,5,7,8,10,11,12}.
I initially wanted to create a parallel vector and insert the (sequential vector) values but I do not know, to which index to insert the values to. But I do know the total size of Vec (which in this case is 7).
Any help is much appreciated.
Kind regards,
Karthik.
This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
Thank you Matt and Blaise. I will try out IS (though I have not it before) (i) Can IS be of different size, on different processors, and still call ISALLGather? (ii) Can IS be passed as row indices to MatZeroRowsColumns? I will look into DMlabels and start a different thread if needed. Best, Karthik. From: Matthew Knepley <[email protected]> Date: Saturday, 10 December 2022 at 02:20 To: Chockalingam, Karthikeyan (STFC,DL,HC) <[email protected]> Cc: Mark Adams <[email protected]>, [email protected] <[email protected]> Subject: Re: [petsc-users] Union of sequential vecs On Fri, Dec 9, 2022 at 6:50 PM Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected]<mailto:[email protected]>> wrote: Thank you Mark and Barry. @Mark Adams<mailto:[email protected]> I follow you for the most part. Shouldn’t R be an MPI Vector? Here it goes: Q[Vec[i]] for all "i" in my local (sequential) "Vec". Compute number of local nonzeros in Q, call it n. //Create a MPI Vector R, with local size n VecCreate(PETSC_COMM_WORLD, &R); VecSetType(R, VECMPI); VecSetSizes(R, n, PETSC_DECIDE); //Populate MPI Vector R with local (sequential) "Vec". VecGetOwnershipRange(R, &istart, &iend); local_size = iend - istart; //The local_size should be ‘n’ right? VecGetArray(R, &values); for (i = 0; i < local_size; i++) { values[i] = Vec[i]; } VecRestoreArray(R, &values); //Scatter R to all processors Vec V_SEQ; VecScatter ctx; VecScatterCreateToAll(R,&ctx,&V_SEQ); //Remove duplicates in V_SEQ How can I use PetscSortRemoveDupsInt to remove duplicates in V_SEQ? Physics behind: I am reading a parallel mesh, and want to mark all the boundary nodes. I use a local (sequential) Vec to store the boundary nodes for each parallel partition. Hence, local Vecs can end up with duplicate node index among them, which I would like to get rid of when I combine all of them together. 1) Blaise is right you should use an IS, not a Vec, to hold node indices. His solution is only a few lines, so I would use it. 2) I would not recommend doing things this way in the first place. PETSc can manage parallel meshes scalably, marking boundaries using DMLabel objects. Thanks, Matt Best, Karthik. From: Mark Adams <[email protected]<mailto:[email protected]>> Date: Friday, 9 December 2022 at 21:08 To: Chockalingam, Karthikeyan (STFC,DL,HC) <[email protected]<mailto:[email protected]>> Cc: Barry Smith <[email protected]<mailto:[email protected]>>, [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Subject: Re: [petsc-users] Union of sequential vecs If your space is pretty compact, eg, (0,12), you could create an MPI vector Q of size 13, say, and each processor can add 1.0 to Q[Vec[i]], for all "i" in my local "Vec". Then each processor can count the number of local nonzeros in Q, call it n, create a new vector, R, with local size n, then set R[i] = global index of the nonzero for each nonzero in Q, i=0:n. Do some sort of vec-scatter-to-all with R to get what you want. Does that work? Mark On Fri, Dec 9, 2022 at 3:25 PM Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected]<mailto:[email protected]>> wrote: That is where I am stuck, I don’t know who to combine them to get Vec = {2,5,7,8,10,11,12}. I just want them in an MPI vector. I finally plan to call VecScatterCreateToAll so that all processor gets a copy. Thank you. Kind regards, Karthik. From: Barry Smith <[email protected]<mailto:[email protected]>> Date: Friday, 9 December 2022 at 20:04 To: Chockalingam, Karthikeyan (STFC,DL,HC) <[email protected]<mailto:[email protected]>> Cc: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Subject: Re: [petsc-users] Union of sequential vecs How are you combining them to get Vec = {2,5,7,8,10,11,12}? Do you want the values to remain on the same MPI rank as before, just in an MPI vector? On Dec 9, 2022, at 2:28 PM, Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected]<mailto:[email protected]>> wrote: Hi, I want to take the union of a set of sequential vectors, each living in a different processor. Say, Vec_Seq1 = {2,5,7} Vec_Seq2 = {5,8,10,11} Vec_Seq3 = {5,2,12}. Finally, get the union of all them Vec = {2,5,7,8,10,11,12}. I initially wanted to create a parallel vector and insert the (sequential vector) values but I do not know, to which index to insert the values to. But I do know the total size of Vec (which in this case is 7). Any help is much appreciated. Kind regards, Karthik. This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses. -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/<http://www.cse.buffalo.edu/~knepley/>
On Mon, Dec 12, 2022 at 10:00 AM Karthikeyan Chockalingam - STFC UKRI < [email protected]> wrote:
Thank you Matt and Blaise.
I will try out IS (though I have not it before)
(i) Can IS be of different size, on different processors, and still call ISALLGather?
Yes.
(ii) Can IS be passed as row indices to MatZeroRowsColumns?
https://petsc.org/main/docs/manualpages/Mat/MatZeroRowsColumnsIS/ Thanks, Matt
I will look into DMlabels and start a different thread if needed.
Best,
Karthik.
*From: *Matthew Knepley <[email protected]> *Date: *Saturday, 10 December 2022 at 02:20 *To: *Chockalingam, Karthikeyan (STFC,DL,HC) < [email protected]> *Cc: *Mark Adams <[email protected]>, [email protected] < [email protected]> *Subject: *Re: [petsc-users] Union of sequential vecs
On Fri, Dec 9, 2022 at 6:50 PM Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected]> wrote:
Thank you Mark and Barry.
@Mark Adams <[email protected]> I follow you for the most part. Shouldn’t R be an MPI Vector?
Here it goes:
Q[Vec[i]] for all "i" in my local (sequential) "Vec".
Compute number of local nonzeros in Q, call it n.
//Create a MPI Vector R, with local size n
VecCreate(PETSC_COMM_WORLD, &R);
VecSetType(R, VECMPI);
VecSetSizes(R, *n*, PETSC_DECIDE);
//Populate MPI Vector R with local (sequential) "Vec".
VecGetOwnershipRange(R, &istart, &iend);
local_size = iend - istart; //The local_size should be ‘*n*’ right?
VecGetArray(R, &values);
*for* (i = 0; i < local_size; i++) {
values[i] = Vec[i];
}
VecRestoreArray(R, &values);
//Scatter R to all processors
Vec V_SEQ;
VecScatter ctx;
VecScatterCreateToAll(R,&ctx,&V_SEQ);
//Remove duplicates in V_SEQ
How can I use PetscSortRemoveDupsInt to remove duplicates in V_SEQ?
Physics behind:
I am reading a parallel mesh, and want to mark all the boundary nodes. I use a local (sequential) Vec to store the boundary nodes for each parallel partition. Hence, local Vecs can end up with duplicate node index among them, which I would like to get rid of when I combine all of them together.
1) Blaise is right you should use an IS, not a Vec, to hold node indices. His solution is only a few lines, so I would use it.
2) I would not recommend doing things this way in the first place. PETSc can manage parallel meshes scalably, marking boundaries using DMLabel objects.
Thanks,
Matt
Best,
Karthik.
*From: *Mark Adams <[email protected]> *Date: *Friday, 9 December 2022 at 21:08 *To: *Chockalingam, Karthikeyan (STFC,DL,HC) < [email protected]> *Cc: *Barry Smith <[email protected]>, [email protected] < [email protected]> *Subject: *Re: [petsc-users] Union of sequential vecs
If your space is pretty compact, eg, (0,12), you could create an MPI vector Q of size 13, say, and each processor can add 1.0 to Q[Vec[i]], for all "i" in my local "Vec".
Then each processor can count the number of local nonzeros in Q, call it n, create a new vector, R, with local size n, then set R[i] = global index of the nonzero for each nonzero in Q, i=0:n.
Do some sort of vec-scatter-to-all with R to get what you want.
Does that work?
Mark
On Fri, Dec 9, 2022 at 3:25 PM Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected]> wrote:
That is where I am stuck, *I don’t know* who to combine them to get Vec = {2,5,7,8,10,11,12}.
I just want them in an MPI vector.
I finally plan to call VecScatterCreateToAll so that all processor gets a copy.
Thank you.
Kind regards,
Karthik.
*From: *Barry Smith <[email protected]> *Date: *Friday, 9 December 2022 at 20:04 *To: *Chockalingam, Karthikeyan (STFC,DL,HC) < [email protected]> *Cc: *[email protected] <[email protected]> *Subject: *Re: [petsc-users] Union of sequential vecs
How are you combining them to get Vec = {2,5,7,8,10,11,12}?
Do you want the values to remain on the same MPI rank as before, just in an MPI vector?
On Dec 9, 2022, at 2:28 PM, Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected]> wrote:
Hi,
I want to take the union of a set of sequential vectors, each living in a different processor.
Say,
Vec_Seq1 = {2,5,7}
Vec_Seq2 = {5,8,10,11}
Vec_Seq3 = {5,2,12}.
Finally, get the union of all them Vec = {2,5,7,8,10,11,12}.
I initially wanted to create a parallel vector and insert the (sequential vector) values but I do not know, to which index to insert the values to. But I do know the total size of Vec (which in this case is 7).
Any help is much appreciated.
Kind regards,
Karthik.
This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
--
What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener
https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
-- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
Ok, so you want the unique list of integers sorted from all the seq vectors on ever MPI rank? VecScatterCreateToAll() to get all values on all ranks (make the sequential vectors MPI vectors instead). create an integer array long enough to hold all of them Use VecGetArray() and a for loop to copy all the values to the integer array, Use PetscSortRemoveDupsInt on the integer array Now each rank has all the desired values.
On Dec 9, 2022, at 3:24 PM, Karthikeyan Chockalingam - STFC UKRI <[email protected]> wrote:
That is where I am stuck, I don’t know who to combine them to get Vec = {2,5,7,8,10,11,12}. I just want them in an MPI vector.
I finally plan to call VecScatterCreateToAll so that all processor gets a copy.
Thank you.
Kind regards, Karthik.
From: Barry Smith <[email protected] <mailto:[email protected]>> Date: Friday, 9 December 2022 at 20:04 To: Chockalingam, Karthikeyan (STFC,DL,HC) <[email protected] <mailto:[email protected]>> Cc: [email protected] <mailto:[email protected]> <[email protected] <mailto:[email protected]>> Subject: Re: [petsc-users] Union of sequential vecs
How are you combining them to get Vec = {2,5,7,8,10,11,12}?
Do you want the values to remain on the same MPI rank as before, just in an MPI vector?
On Dec 9, 2022, at 2:28 PM, Karthikeyan Chockalingam - STFC UKRI via petsc-users <[email protected] <mailto:[email protected]>> wrote:
Hi,
I want to take the union of a set of sequential vectors, each living in a different processor.
Say, Vec_Seq1 = {2,5,7} Vec_Seq2 = {5,8,10,11} Vec_Seq3 = {5,2,12}.
Finally, get the union of all them Vec = {2,5,7,8,10,11,12}.
I initially wanted to create a parallel vector and insert the (sequential vector) values but I do not know, to which index to insert the values to. But I do know the total size of Vec (which in this case is 7).
Any help is much appreciated.
Kind regards, Karthik.
This email and any attachments are intended solely for the use of the named recipients. If you are not the intended recipient you must not use, disclose, copy or distribute this email or any of its attachments and should notify the sender immediately and delete this email from your system. UK Research and Innovation (UKRI) has taken every reasonable precaution to minimise risk of this email or any attachments containing viruses or malware but the recipient should carry out its own virus and malware checks before opening the attachments. UKRI does not accept any liability for any losses or damages which the recipient may sustain due to presence of any viruses.
participants (5)
-
Barry Smith -
Blaise Bourdin -
Karthikeyan Chockalingam - STFC UKRI -
Mark Adams -
Matthew Knepley