Dear Pierre, To complete my last post, in fact, the initial code (playing with the LGMap) was correct. It was my test case that was wrong. Once fixed according to your suggestion, everything turns out to be OK. I am nevertheless wondering if this IS embeding according to the global numbering should not be a native PTESc's feature ? Thank you again, Nicolas -- Nicolas Tardieu Ing PhD Computational Mechanics EDF - R&D Dpt ERMES PARIS-SACLAY, FRANCE ________________________________ De : TARDIEU Nicolas <[email protected]> Envoyé : jeudi 20 octobre 2022 11:45 À : [email protected] <[email protected]> Cc : [email protected] <[email protected]> Objet : RE: [petsc-users] Trouble with ISEmbed Dear Pierre, You fixed the problem! Thank you warmly for your precious help. Best regards, Nicolas -- Nicolas Tardieu Ing PhD Computational Mechanics EDF - R&D Dpt ERMES PARIS-SACLAY, FRANCE ________________________________ De : [email protected] <[email protected]> Envoyé : mercredi 19 octobre 2022 22:22 À : TARDIEU Nicolas <[email protected]> Cc : [email protected] <[email protected]> Objet : Re: [petsc-users] Trouble with ISEmbed Sorry, I’m not very proficient in petsc4py, and there are a bunch of interfaces missing, e.g., ISShift(), so it may not be optimal, but I hope you’ll understand. First, you’ll need to regenerate the .bin by uncommenting the proper part of the code. That is because you were initially generating a 20x20 matrix, with 4 fields per unknown. That’s 5 unknowns, and so, with two processes, 10 rows per process is not consistent as 10/4 is not an integer — I don’t know how to force, in petsc4py, the local size to 12 on process #0 and 8 on process #1. The modified code generates a 16x16 matrices so it remains consistent. If you then run the first part of the program, you’ll get both B_uu and B_pp from B instead of A, with one, two, or four processes. Again, that should work for arbitrary number of processes, you just need to be careful that your local dimensions are consistent with the number of fields. Thanks, Pierre On 19 Oct 2022, at 5:01 PM, Pierre Jolivet <[email protected]<mailto:[email protected]>> wrote: On 19 Oct 2022, at 4:32 PM, TARDIEU Nicolas <[email protected]<mailto:[email protected]>> wrote: Dear Pierre, Thank you very much for your answer. I have the same explanation as you for the code I sent. But what I would like to do is the following : I have the full matrix A with fields u, p and t (which are interlaced in the real application). I want to extract B=A(u+p, u+p). *Then* I would like to extract the (u, u) block from B - let us call it B_uu. In fact, B_uu=A_uu but I really need to do the extraction from B. And I am missing something since I have to play with different numberings when switching the IS from A to B. Is it clear enough ???? That’s cristal clear. If the fields are interlaced, that’s actually easier to do, because you preserve the distribution, and there is less data movement. I’ll try to fix your code in the case where the fields are interlaced if now one gives you another answer in the meantime. Thanks, Pierre Regards, Nicolas -- Nicolas Tardieu Ing PhD Computational Mechanics EDF - R&D Dpt ERMES PARIS-SACLAY, FRANCE ________________________________ De : [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Envoyé : mercredi 19 octobre 2022 14:51 À : TARDIEU Nicolas <[email protected]<mailto:[email protected]>> Cc : [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Objet : Re: [petsc-users] Trouble with ISEmbed On two processes, you have a different distribution for u and u+p. IS Object: 2 MPI processes type: general [0] Number of indices in set 5 [0] 0 0 [0] 1 1 [0] 2 2 [0] 3 3 [0] 4 4 [1] Number of indices in set 5 [1] 0 5 [1] 1 6 [1] 2 7 [1] 3 8 [1] 4 9 IS Object: 2 MPI processes type: general [0] Number of indices in set 8 [0] 0 0 [0] 1 1 [0] 2 2 [0] 3 3 [0] 4 4 [0] 5 5 [0] 6 6 [0] 7 7 [1] Number of indices in set 7 [1] 0 8 [1] 1 9 [1] 2 10 [1] 3 11 [1] 4 12 [1] 5 13 [1] 6 14 ISEmbed() only works on local indices, so when you embed u into u+p, on the second process, you miss the row/column indices 5, 6, 7 of B = A(u+p, u+p). Thus, you end up with a matrix of dimension size(u) - 3 = 10 - 3 = 7, with just the row/column indices 8 and 9 being selected by the second process. What is it that you want to do exactly? Play with ISEmbed(), or get A(u, u) without using A but B instead? Thanks, Pierre
On 19 Oct 2022, at 12:00 PM, TARDIEU Nicolas via petsc-users <[email protected]<mailto:[email protected]>> wrote:
Dear PETSc Team,
I am trying to use IS embeding in parallel. In order to (try to) understand how it works, I have built a simple example, attached to this email.
I consider a 20X20 matrix. The dof (u, p, t) in global numbering are the following : u: 0..9 p: 10..14 t: 15..19
I have defined 4 IS to describe the dof u, p, t and the agglomeration of u and p, called up. I first extract the submatrix matrix(up,up), then I would like to extract from it the (u,u) block.
The example runs OK in sequential but I do not obtain the (u,u) block on 2 processes.
I have a mistake in the build of the sub-IS but I cannot find it for days.
Best regards, Nicolas -- Nicolas Tardieu Ing PhD Computational Mechanics EDF - R&D Dpt ERMES PARIS-SACLAY, FRANCE
Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message. Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. <is_pb.tgz>
Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message. Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free. Ce message et toutes les pièces jointes (ci-après le 'Message') sont établis à l'intention exclusive des destinataires et les informations qui y figurent sont strictement confidentielles. Toute utilisation de ce Message non conforme à sa destination, toute diffusion ou toute publication totale ou partielle, est interdite sauf autorisation expresse. Si vous n'êtes pas le destinataire de ce Message, il vous est interdit de le copier, de le faire suivre, de le divulguer ou d'en utiliser tout ou partie. Si vous avez reçu ce Message par erreur, merci de le supprimer de votre système, ainsi que toutes ses copies, et de n'en garder aucune trace sur quelque support que ce soit. Nous vous remercions également d'en avertir immédiatement l'expéditeur par retour du message. Il est impossible de garantir que les communications par messagerie électronique arrivent en temps utile, sont sécurisées ou dénuées de toute erreur ou virus. ____________________________________________________ This message and any attachments (the 'Message') are intended solely for the addressees. The information contained in this Message is confidential. Any use of information contained in this Message not in accord with its purpose, any dissemination or disclosure, either whole or partial, is prohibited except formal approval. If you are not the addressee, you may not copy, forward, disclose or use any part of it. If you have received this message in error, please delete it and all copies from your system and notify the sender immediately by return message. E-mail communication cannot be guaranteed to be timely secure, error or virus-free.