March 2021 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Argonne GPU Virtual Hackathon - Accepted
by Jed Brown 12 Mar '21

12 Mar '21

Barry Smith <bsmith(a)petsc.dev> writes: > I think we should start porting the PetscFE infrastructure, numerical integrations, vector and matrix assembly to GPUs soon. It is dog slow on CPUs and should be able to deliver higher performance on GPUs. IMO, this comes via interfaces to libCEED, not rolling yet another way to invoke quadrature routines on GPUs. DMPlex setup and distribution could conceivably be ported to GPUs, but it would be a monumental task and that stuff is usually done once and intermediate structures don't necessarily fit in device memory even when they fit comfortably in DRAM. Way too big of a task for one hackathon.

1 0

Re: [petsc-dev] Argonne GPU Virtual Hackathon - Accepted
by Barry Smith 12 Mar '21

12 Mar '21

Hong Thanks. What else is needed to get all of GAMG setup onto the GPU and running as well as it can? Any more GPU kernels? Or is just tweaking the infrastucture, the kernels are already done? Will we see pure GPU GAMG soon? I think we should start porting the PetscFE infrastructure, numerical integrations, vector and matrix assembly to GPUs soon. It is dog slow on CPUs and should be able to deliver higher performance on GPUs. Others? Barry And no, this is not the time for arguing GPUs suck and some things are best done on CPUs or that DOE should not have bought GPU machines. It is about getting the best performance we can from a set of systems we will have available. > On Mar 12, 2021, at 5:43 PM, Zhang, Hong <hongzhang(a)anl.gov> wrote: > > > >> On Mar 12, 2021, at 5:25 PM, Barry Smith <bsmith(a)petsc.dev <mailto:[email protected]>> wrote: >> >> >> Jed, >> >> Thanks for the insight. >> >> Maybe Hong and his Ellpack format? Or his independent set algorithm? > > These two features are currently functional on NVIDIA GPUs. Neither needs extensive development or refactorization, thus not suitable for this Argonne GPU Hackathon. But we do need reviewers to look at the two MRs and move forward. > > It is also worth mentioning that the CUDA-based maximal independent set algorithm was developed during last GPU Hackathon at ORNL. Richard, Junchao and I attended that one. It was a great experience. The commitment is extremely intensive for both the mentors and the attendees. > > Thanks, > Hong > >> >> Maybe Stefano and his COO matrix assembly on GPUs? >> >> Others? >> >> Barry >> >> >>> On Mar 12, 2021, at 4:37 PM, Jed Brown <jed(a)jedbrown.org <mailto:[email protected]>> wrote: >>> >>> I helped with one of these a couple years ago. It's important to go in with a well-defined problem and mini-app. If "PETSc" is the topic, then you should start with a representative benchmark problem that runs in no more than a couple minutes. It could be two problems, one that we think is good and one that we think is weak. The format is good for incrementally profiling and refactoring existing code. Probably less so for doing careful implementations of genuinely novel algorithms. (And if you plan to do that, make sure there's a way that the one person focused on that lift doesn't block the rest of the team.) >>> >>> Barry Smith <bsmith(a)petsc.dev <mailto:[email protected]>> writes: >>> >>>> Folks, >>>> >>>> Get was able to get us into the ANL GPU hackathon that will use the "theta" GPU system at ANL (that has NVIDIA GPUs). ALCF accounts are needed for the hackathon (but can be created). Registration information is below. If we go with specific goals it could be a useful experience. >>>> >>>> Barry >>>> >>>> >>>> >>>>> Begin forwarded message: >>>>> >>>>> From: Andi Moore <amoore(a)nvidia.com <mailto:[email protected]>> >>>>> Subject: Argonne GPU Virtual Hackathon - Accepted >>>>> Date: March 11, 2021 at 5:09:02 PM CST >>>>> To: "d.getnet(a)gmail.com <mailto:[email protected]>" <d.getnet(a)gmail.com <mailto:[email protected]>>, "gbetrie(a)anl.gov <mailto:[email protected]>" <gbetrie(a)anl.gov <mailto:[email protected]>>, "bsmith(a)petsc.dev <mailto:[email protected]>" <bsmith(a)petsc.dev <mailto:[email protected]>> >>>>> Cc: "Ghadar, Yasaman" <ghadar(a)anl.gov <mailto:[email protected]>>, Julia Levites <jlevites(a)nvidia.com <mailto:[email protected]>> >>>>> Message-Id: <MW3PR12MB4588BF988609B8F7A49F6A21B3909(a)MW3PR12MB4588.namprd12.prod.outlook.com <mailto:MW3PR12MB4588BF988609B8F7A49F6A21B3909@MW3PR12MB4588.namprd12.prod.outlook.com>> >>>>> >>>>> Dear Team Barry Smith, >>>>> >>>>> Congratulations! Your team has been selected to participate in the Argonne GPU Virtual Hackathon. >>>>> >>>>> The next steps: >>>>> >>>>> Registration >>>>> Each team member is required to register for the event. Please click here <https://www.gpuhackathons.org/form/gpu-event-registration <https://www.gpuhackathons.org/form/gpu-event-registration>> to register and select the Hackathon from the drop-down menu. Please register as soon as possible. >>>>> >>>>> Mentors >>>>> Mentors are being selected for each team based on your research fields and coding needs. An introductory call for teams and mentors has been scheduled for Wednesday, April 7th, details below. >>>>> >>>>> System Access >>>>> Request an ALCF Computer User Account <https://accounts.alcf.anl.gov/ <https://accounts.alcf.anl.gov/>> if you do not currently have one. If you have an ALCF account, but it is currently inactive, reactivate <https://accounts.alcf.anl.gov/#!/accountReactivate <https://accounts.alcf.anl.gov/#!/accountReactivate>> your account. >>>>> Specify the following in the account request/reactivation form: >>>>> Project Name: gpu_hack >>>>> Principal Investigator (PI): Yasaman Ghadar >>>>> >>>>> Contact accounts(a)alcf.anl.gov <mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>> if you have questions about your ALCF account. >>>>> >>>>> Additional information >>>>> Review the GPU Hackathon Virtual Attendee Guide <https://docs.google.com/document/d/1M7vacg0GLg38gsJYHOg2tM_M0QArOSCrPiLaQER… <https://docs.google.com/document/d/1M7vacg0GLg38gsJYHOg2tM_M0QArOSCrPiLaQER…>> >>>>> Review the technical resources website <https://www.gpuhackathons.org/technical-resources <https://www.gpuhackathons.org/technical-resources>> which will help you prepare for the hackathon. The page includes information about profiling tools, compilers, libraries, programming models and more. >>>>> >>>>> Slack >>>>> All communication for the event will occur through the Slack workspace >>>>> Please join <https://join.slack.com/t/argonnehack2021/shared_invite/zt-nfio2vwz-f0UeYdXV… <https://join.slack.com/t/argonnehack2021/shared_invite/zt-nfio2vwz-f0UeYdXV…>> the Slack workspace >>>>> >>>>> Agenda - All calendar invites are attached >>>>> Argonne GPU Hackathon Team and Mentor Meeting: 2:00PM-3:00PM CST – April 7, 2021 Zoom Link <https://zoom.us/j/93527581258?pwd=S01GZVAxS3hLVCsvc0wyR2s2MHJEZz09 <https://zoom.us/j/93527581258?pwd=S01GZVAxS3hLVCsvc0wyR2s2MHJEZz09>> >>>>> >>>>> Argonne GPU Hackathon Day 1: 9:00AM-5:00PM CST – April 20, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09 <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09>> >>>>> >>>>> Argonne GPU Hackathon Day 2: 9:00AM-5:00PM CST April 27, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09 <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09>> >>>>> >>>>> Argonne GPU Hackathon Day 3: 9:00AM-5:00PM CST April 28, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09 <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09>> >>>>> >>>>> Argonne GPU Hackathon Day 4: 9:00AM-5:30PM CST April 29, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09 <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09>> >>>>> >>>>> >>>>> Please feel free to ask any questions you may have. Thank you all for your interest and outstanding applications! >>>>> >>>>> Best regards, >>>>> The GPU Hackathon Team >> >

1 0

Re: [petsc-dev] Argonne GPU Virtual Hackathon - Accepted
by Zhang, Hong 12 Mar '21

12 Mar '21

On Mar 12, 2021, at 5:25 PM, Barry Smith <bsmith(a)petsc.dev<mailto:[email protected]>> wrote: Jed, Thanks for the insight. Maybe Hong and his Ellpack format? Or his independent set algorithm? These two features are currently functional on NVIDIA GPUs. Neither needs extensive development or refactorization, thus not suitable for this Argonne GPU Hackathon. But we do need reviewers to look at the two MRs and move forward. It is also worth mentioning that the CUDA-based maximal independent set algorithm was developed during last GPU Hackathon at ORNL. Richard, Junchao and I attended that one. It was a great experience. The commitment is extremely intensive for both the mentors and the attendees. Thanks, Hong Maybe Stefano and his COO matrix assembly on GPUs? Others? Barry On Mar 12, 2021, at 4:37 PM, Jed Brown <jed(a)jedbrown.org<mailto:[email protected]>> wrote: I helped with one of these a couple years ago. It's important to go in with a well-defined problem and mini-app. If "PETSc" is the topic, then you should start with a representative benchmark problem that runs in no more than a couple minutes. It could be two problems, one that we think is good and one that we think is weak. The format is good for incrementally profiling and refactoring existing code. Probably less so for doing careful implementations of genuinely novel algorithms. (And if you plan to do that, make sure there's a way that the one person focused on that lift doesn't block the rest of the team.) Barry Smith <bsmith(a)petsc.dev<mailto:[email protected]>> writes: Folks, Get was able to get us into the ANL GPU hackathon that will use the "theta" GPU system at ANL (that has NVIDIA GPUs). ALCF accounts are needed for the hackathon (but can be created). Registration information is below. If we go with specific goals it could be a useful experience. Barry Begin forwarded message: From: Andi Moore <amoore(a)nvidia.com<mailto:[email protected]>> Subject: Argonne GPU Virtual Hackathon - Accepted Date: March 11, 2021 at 5:09:02 PM CST To: "d.getnet(a)gmail.com<mailto:[email protected]>" <d.getnet(a)gmail.com<mailto:[email protected]>>, "gbetrie(a)anl.gov<mailto:[email protected]>" <gbetrie(a)anl.gov<mailto:[email protected]>>, "bsmith(a)petsc.dev<mailto:[email protected]>" <bsmith(a)petsc.dev<mailto:[email protected]>> Cc: "Ghadar, Yasaman" <ghadar(a)anl.gov<mailto:[email protected]>>, Julia Levites <jlevites(a)nvidia.com<mailto:[email protected]>> Message-Id: <MW3PR12MB4588BF988609B8F7A49F6A21B3909(a)MW3PR12MB4588.namprd12.prod.outlook.com<mailto:MW3PR12MB4588BF988609B8F7A49F6A21B3909@MW3PR12MB4588.namprd12.prod.outlook.com>> Dear Team Barry Smith, Congratulations! Your team has been selected to participate in the Argonne GPU Virtual Hackathon. The next steps: Registration Each team member is required to register for the event. Please click here <https://www.gpuhackathons.org/form/gpu-event-registration> to register and select the Hackathon from the drop-down menu. Please register as soon as possible. Mentors Mentors are being selected for each team based on your research fields and coding needs. An introductory call for teams and mentors has been scheduled for Wednesday, April 7th, details below. System Access Request an ALCF Computer User Account <https://accounts.alcf.anl.gov/> if you do not currently have one. If you have an ALCF account, but it is currently inactive, reactivate <https://accounts.alcf.anl.gov/#!/accountReactivate> your account. Specify the following in the account request/reactivation form: Project Name: gpu_hack Principal Investigator (PI): Yasaman Ghadar Contact accounts(a)alcf.anl.gov<mailto:[email protected]> <mailto:[email protected]> if you have questions about your ALCF account. Additional information Review the GPU Hackathon Virtual Attendee Guide <https://docs.google.com/document/d/1M7vacg0GLg38gsJYHOg2tM_M0QArOSCrPiLaQER…> Review the technical resources website <https://www.gpuhackathons.org/technical-resources> which will help you prepare for the hackathon. The page includes information about profiling tools, compilers, libraries, programming models and more. Slack All communication for the event will occur through the Slack workspace Please join <https://join.slack.com/t/argonnehack2021/shared_invite/zt-nfio2vwz-f0UeYdXV…> the Slack workspace Agenda - All calendar invites are attached Argonne GPU Hackathon Team and Mentor Meeting: 2:00PM-3:00PM CST – April 7, 2021 Zoom Link <https://zoom.us/j/93527581258?pwd=S01GZVAxS3hLVCsvc0wyR2s2MHJEZz09> Argonne GPU Hackathon Day 1: 9:00AM-5:00PM CST – April 20, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09> Argonne GPU Hackathon Day 2: 9:00AM-5:00PM CST April 27, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09> Argonne GPU Hackathon Day 3: 9:00AM-5:00PM CST April 28, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09> Argonne GPU Hackathon Day 4: 9:00AM-5:30PM CST April 29, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09> Please feel free to ask any questions you may have. Thank you all for your interest and outstanding applications! Best regards, The GPU Hackathon Team

1 0

Re: [petsc-dev] Argonne GPU Virtual Hackathon - Accepted
by Jed Brown 12 Mar '21

12 Mar '21

Sure, AMG setup is another (especially if Mark/someone could do some legwork in advance; this might be a rabbit hole). Deluxe BDDC if Stefano is there. We include a libCEED-based PETSc miniapp looking at assembly and MG bottlenecks (using mostly matrix-free data structures). Barry Smith <bsmith(a)petsc.dev> writes: > Jed, > > Thanks for the insight. > > Maybe Hong and his Ellpack format? Or his independent set algorithm? > > Maybe Stefano and his COO matrix assembly on GPUs? > > Others? > > Barry > > >> On Mar 12, 2021, at 4:37 PM, Jed Brown <jed(a)jedbrown.org> wrote: >> >> I helped with one of these a couple years ago. It's important to go in with a well-defined problem and mini-app. If "PETSc" is the topic, then you should start with a representative benchmark problem that runs in no more than a couple minutes. It could be two problems, one that we think is good and one that we think is weak. The format is good for incrementally profiling and refactoring existing code. Probably less so for doing careful implementations of genuinely novel algorithms. (And if you plan to do that, make sure there's a way that the one person focused on that lift doesn't block the rest of the team.) >> >> Barry Smith <bsmith(a)petsc.dev <mailto:[email protected]>> writes: >> >>> Folks, >>> >>> Get was able to get us into the ANL GPU hackathon that will use the "theta" GPU system at ANL (that has NVIDIA GPUs). ALCF accounts are needed for the hackathon (but can be created). Registration information is below. If we go with specific goals it could be a useful experience. >>> >>> Barry >>> >>> >>> >>>> Begin forwarded message: >>>> >>>> From: Andi Moore <amoore(a)nvidia.com> >>>> Subject: Argonne GPU Virtual Hackathon - Accepted >>>> Date: March 11, 2021 at 5:09:02 PM CST >>>> To: "d.getnet(a)gmail.com" <d.getnet(a)gmail.com>, "gbetrie(a)anl.gov" <gbetrie(a)anl.gov>, "bsmith(a)petsc.dev" <bsmith(a)petsc.dev> >>>> Cc: "Ghadar, Yasaman" <ghadar(a)anl.gov>, Julia Levites <jlevites(a)nvidia.com> >>>> Message-Id: <MW3PR12MB4588BF988609B8F7A49F6A21B3909(a)MW3PR12MB4588.namprd12.prod.outlook.com> >>>> >>>> Dear Team Barry Smith, >>>> >>>> Congratulations! Your team has been selected to participate in the Argonne GPU Virtual Hackathon. >>>> >>>> The next steps: >>>> >>>> Registration >>>> Each team member is required to register for the event. Please click here <https://www.gpuhackathons.org/form/gpu-event-registration <https://www.gpuhackathons.org/form/gpu-event-registration>> to register and select the Hackathon from the drop-down menu. Please register as soon as possible. >>>> >>>> Mentors >>>> Mentors are being selected for each team based on your research fields and coding needs. An introductory call for teams and mentors has been scheduled for Wednesday, April 7th, details below. >>>> >>>> System Access >>>> Request an ALCF Computer User Account <https://accounts.alcf.anl.gov/ <https://accounts.alcf.anl.gov/>> if you do not currently have one. If you have an ALCF account, but it is currently inactive, reactivate <https://accounts.alcf.anl.gov/#!/accountReactivate <https://accounts.alcf.anl.gov/#!/accountReactivate>> your account. >>>> Specify the following in the account request/reactivation form: >>>> Project Name: gpu_hack >>>> Principal Investigator (PI): Yasaman Ghadar >>>> >>>> Contact accounts(a)alcf.anl.gov <mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>> if you have questions about your ALCF account. >>>> >>>> Additional information >>>> Review the GPU Hackathon Virtual Attendee Guide <https://docs.google.com/document/d/1M7vacg0GLg38gsJYHOg2tM_M0QArOSCrPiLaQER… <https://docs.google.com/document/d/1M7vacg0GLg38gsJYHOg2tM_M0QArOSCrPiLaQER…>> >>>> Review the technical resources website <https://www.gpuhackathons.org/technical-resources <https://www.gpuhackathons.org/technical-resources>> which will help you prepare for the hackathon. The page includes information about profiling tools, compilers, libraries, programming models and more. >>>> >>>> Slack >>>> All communication for the event will occur through the Slack workspace >>>> Please join <https://join.slack.com/t/argonnehack2021/shared_invite/zt-nfio2vwz-f0UeYdXV… <https://join.slack.com/t/argonnehack2021/shared_invite/zt-nfio2vwz-f0UeYdXV…>> the Slack workspace >>>> >>>> Agenda - All calendar invites are attached >>>> Argonne GPU Hackathon Team and Mentor Meeting: 2:00PM-3:00PM CST – April 7, 2021 Zoom Link <https://zoom.us/j/93527581258?pwd=S01GZVAxS3hLVCsvc0wyR2s2MHJEZz09 <https://zoom.us/j/93527581258?pwd=S01GZVAxS3hLVCsvc0wyR2s2MHJEZz09>> >>>> >>>> Argonne GPU Hackathon Day 1: 9:00AM-5:00PM CST – April 20, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09 <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09>> >>>> >>>> Argonne GPU Hackathon Day 2: 9:00AM-5:00PM CST April 27, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09 <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09>> >>>> >>>> Argonne GPU Hackathon Day 3: 9:00AM-5:00PM CST April 28, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09 <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09>> >>>> >>>> Argonne GPU Hackathon Day 4: 9:00AM-5:30PM CST April 29, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09 <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09>> >>>> >>>> >>>> Please feel free to ask any questions you may have. Thank you all for your interest and outstanding applications! >>>> >>>> Best regards, >>>> The GPU Hackathon Team

1 0

Re: [petsc-dev] Argonne GPU Virtual Hackathon - Accepted
by Barry Smith 12 Mar '21

12 Mar '21

Jed, Thanks for the insight. Maybe Hong and his Ellpack format? Or his independent set algorithm? Maybe Stefano and his COO matrix assembly on GPUs? Others? Barry > On Mar 12, 2021, at 4:37 PM, Jed Brown <jed(a)jedbrown.org> wrote: > > I helped with one of these a couple years ago. It's important to go in with a well-defined problem and mini-app. If "PETSc" is the topic, then you should start with a representative benchmark problem that runs in no more than a couple minutes. It could be two problems, one that we think is good and one that we think is weak. The format is good for incrementally profiling and refactoring existing code. Probably less so for doing careful implementations of genuinely novel algorithms. (And if you plan to do that, make sure there's a way that the one person focused on that lift doesn't block the rest of the team.) > > Barry Smith <bsmith(a)petsc.dev <mailto:[email protected]>> writes: > >> Folks, >> >> Get was able to get us into the ANL GPU hackathon that will use the "theta" GPU system at ANL (that has NVIDIA GPUs). ALCF accounts are needed for the hackathon (but can be created). Registration information is below. If we go with specific goals it could be a useful experience. >> >> Barry >> >> >> >>> Begin forwarded message: >>> >>> From: Andi Moore <amoore(a)nvidia.com> >>> Subject: Argonne GPU Virtual Hackathon - Accepted >>> Date: March 11, 2021 at 5:09:02 PM CST >>> To: "d.getnet(a)gmail.com" <d.getnet(a)gmail.com>, "gbetrie(a)anl.gov" <gbetrie(a)anl.gov>, "bsmith(a)petsc.dev" <bsmith(a)petsc.dev> >>> Cc: "Ghadar, Yasaman" <ghadar(a)anl.gov>, Julia Levites <jlevites(a)nvidia.com> >>> Message-Id: <MW3PR12MB4588BF988609B8F7A49F6A21B3909(a)MW3PR12MB4588.namprd12.prod.outlook.com> >>> >>> Dear Team Barry Smith, >>> >>> Congratulations! Your team has been selected to participate in the Argonne GPU Virtual Hackathon. >>> >>> The next steps: >>> >>> Registration >>> Each team member is required to register for the event. Please click here <https://www.gpuhackathons.org/form/gpu-event-registration <https://www.gpuhackathons.org/form/gpu-event-registration>> to register and select the Hackathon from the drop-down menu. Please register as soon as possible. >>> >>> Mentors >>> Mentors are being selected for each team based on your research fields and coding needs. An introductory call for teams and mentors has been scheduled for Wednesday, April 7th, details below. >>> >>> System Access >>> Request an ALCF Computer User Account <https://accounts.alcf.anl.gov/ <https://accounts.alcf.anl.gov/>> if you do not currently have one. If you have an ALCF account, but it is currently inactive, reactivate <https://accounts.alcf.anl.gov/#!/accountReactivate <https://accounts.alcf.anl.gov/#!/accountReactivate>> your account. >>> Specify the following in the account request/reactivation form: >>> Project Name: gpu_hack >>> Principal Investigator (PI): Yasaman Ghadar >>> >>> Contact accounts(a)alcf.anl.gov <mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>> if you have questions about your ALCF account. >>> >>> Additional information >>> Review the GPU Hackathon Virtual Attendee Guide <https://docs.google.com/document/d/1M7vacg0GLg38gsJYHOg2tM_M0QArOSCrPiLaQER… <https://docs.google.com/document/d/1M7vacg0GLg38gsJYHOg2tM_M0QArOSCrPiLaQER…>> >>> Review the technical resources website <https://www.gpuhackathons.org/technical-resources <https://www.gpuhackathons.org/technical-resources>> which will help you prepare for the hackathon. The page includes information about profiling tools, compilers, libraries, programming models and more. >>> >>> Slack >>> All communication for the event will occur through the Slack workspace >>> Please join <https://join.slack.com/t/argonnehack2021/shared_invite/zt-nfio2vwz-f0UeYdXV… <https://join.slack.com/t/argonnehack2021/shared_invite/zt-nfio2vwz-f0UeYdXV…>> the Slack workspace >>> >>> Agenda - All calendar invites are attached >>> Argonne GPU Hackathon Team and Mentor Meeting: 2:00PM-3:00PM CST – April 7, 2021 Zoom Link <https://zoom.us/j/93527581258?pwd=S01GZVAxS3hLVCsvc0wyR2s2MHJEZz09 <https://zoom.us/j/93527581258?pwd=S01GZVAxS3hLVCsvc0wyR2s2MHJEZz09>> >>> >>> Argonne GPU Hackathon Day 1: 9:00AM-5:00PM CST – April 20, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09 <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09>> >>> >>> Argonne GPU Hackathon Day 2: 9:00AM-5:00PM CST April 27, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09 <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09>> >>> >>> Argonne GPU Hackathon Day 3: 9:00AM-5:00PM CST April 28, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09 <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09>> >>> >>> Argonne GPU Hackathon Day 4: 9:00AM-5:30PM CST April 29, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09 <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09>> >>> >>> >>> Please feel free to ask any questions you may have. Thank you all for your interest and outstanding applications! >>> >>> Best regards, >>> The GPU Hackathon Team

1 0

Re: [petsc-dev] Fwd: Argonne GPU Virtual Hackathon - Accepted
by Jed Brown 12 Mar '21

12 Mar '21

I helped with one of these a couple years ago. It's important to go in with a well-defined problem and mini-app. If "PETSc" is the topic, then you should start with a representative benchmark problem that runs in no more than a couple minutes. It could be two problems, one that we think is good and one that we think is weak. The format is good for incrementally profiling and refactoring existing code. Probably less so for doing careful implementations of genuinely novel algorithms. (And if you plan to do that, make sure there's a way that the one person focused on that lift doesn't block the rest of the team.) Barry Smith <bsmith(a)petsc.dev> writes: > Folks, > > Get was able to get us into the ANL GPU hackathon that will use the "theta" GPU system at ANL (that has NVIDIA GPUs). ALCF accounts are needed for the hackathon (but can be created). Registration information is below. If we go with specific goals it could be a useful experience. > > Barry > > > >> Begin forwarded message: >> >> From: Andi Moore <amoore(a)nvidia.com> >> Subject: Argonne GPU Virtual Hackathon - Accepted >> Date: March 11, 2021 at 5:09:02 PM CST >> To: "d.getnet(a)gmail.com" <d.getnet(a)gmail.com>, "gbetrie(a)anl.gov" <gbetrie(a)anl.gov>, "bsmith(a)petsc.dev" <bsmith(a)petsc.dev> >> Cc: "Ghadar, Yasaman" <ghadar(a)anl.gov>, Julia Levites <jlevites(a)nvidia.com> >> Message-Id: <MW3PR12MB4588BF988609B8F7A49F6A21B3909(a)MW3PR12MB4588.namprd12.prod.outlook.com> >> >> Dear Team Barry Smith, >> >> Congratulations! Your team has been selected to participate in the Argonne GPU Virtual Hackathon. >> >> The next steps: >> >> Registration >> Each team member is required to register for the event. Please click here <https://www.gpuhackathons.org/form/gpu-event-registration> to register and select the Hackathon from the drop-down menu. Please register as soon as possible. >> >> Mentors >> Mentors are being selected for each team based on your research fields and coding needs. An introductory call for teams and mentors has been scheduled for Wednesday, April 7th, details below. >> >> System Access >> Request an ALCF Computer User Account <https://accounts.alcf.anl.gov/> if you do not currently have one. If you have an ALCF account, but it is currently inactive, reactivate <https://accounts.alcf.anl.gov/#!/accountReactivate> your account. >> Specify the following in the account request/reactivation form: >> Project Name: gpu_hack >> Principal Investigator (PI): Yasaman Ghadar >> >> Contact accounts(a)alcf.anl.gov <mailto:[email protected]> if you have questions about your ALCF account. >> >> Additional information >> Review the GPU Hackathon Virtual Attendee Guide <https://docs.google.com/document/d/1M7vacg0GLg38gsJYHOg2tM_M0QArOSCrPiLaQER…> >> Review the technical resources website <https://www.gpuhackathons.org/technical-resources> which will help you prepare for the hackathon. The page includes information about profiling tools, compilers, libraries, programming models and more. >> >> Slack >> All communication for the event will occur through the Slack workspace >> Please join <https://join.slack.com/t/argonnehack2021/shared_invite/zt-nfio2vwz-f0UeYdXV…> the Slack workspace >> >> Agenda - All calendar invites are attached >> Argonne GPU Hackathon Team and Mentor Meeting: 2:00PM-3:00PM CST – April 7, 2021 Zoom Link <https://zoom.us/j/93527581258?pwd=S01GZVAxS3hLVCsvc0wyR2s2MHJEZz09> >> >> Argonne GPU Hackathon Day 1: 9:00AM-5:00PM CST – April 20, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09> >> >> Argonne GPU Hackathon Day 2: 9:00AM-5:00PM CST April 27, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09> >> >> Argonne GPU Hackathon Day 3: 9:00AM-5:00PM CST April 28, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09> >> >> Argonne GPU Hackathon Day 4: 9:00AM-5:30PM CST April 29, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09> >> >> >> Please feel free to ask any questions you may have. Thank you all for your interest and outstanding applications! >> >> Best regards, >> The GPU Hackathon Team >> >>

1 0

Re: [petsc-dev] Petsc "make test" have more failures for --with-openmp=1
by Eric Chamberland 12 Mar '21

12 Mar '21

Hi Pierre, I now have a docker container reproducing the problems here. Actually, if I look at snes_tutorials-ex12_quad_singular_hpddm it fails like this: not ok snes_tutorials-ex12_quad_singular_hpddm # Error code: 59 # Initial guess # L_2 Error: 0.00803099 # Initial Residual # L_2 Residual: 1.09057 # Au - b = Au + F(0) # Linear L_2 Residual: 1.09057 # [d470c54ce086:14127] Read -1, expected 4096, errno = 1 # [d470c54ce086:14128] Read -1, expected 4096, errno = 1 # [d470c54ce086:14129] Read -1, expected 4096, errno = 1 # [3]PETSC ERROR: ------------------------------------------------------------------------ # [3]PETSC ERROR: Caught signal number 11 SEGV: Segmentation Violation, probably memory access out of range # [3]PETSC ERROR: Try option -start_in_debugger or -on_error_attach_debugger # [3]PETSC ERROR: or see https://www.mcs.anl.gov/petsc/documentation/faq.html#valgrind # [3]PETSC ERROR: or try http://valgrind.org on GNU/linux and Apple Mac OS X to find memory corruption errors # [3]PETSC ERROR: likely location of problem given in stack below # [3]PETSC ERROR: --------------------- Stack Frames ------------------------------------ # [3]PETSC ERROR: Note: The EXACT line numbers in the stack are not available, # [3]PETSC ERROR: INSTEAD the line number of the start of the function # [3]PETSC ERROR: is given. # [3]PETSC ERROR: [3] buildTwo line 987 /opt/petsc-main/include/HPDDM_schwarz.hpp # [3]PETSC ERROR: [3] next line 1130 /opt/petsc-main/include/HPDDM_schwarz.hpp # [3]PETSC ERROR: --------------------- Error Message -------------------------------------------------------------- # [3]PETSC ERROR: Signal received # [3]PETSC ERROR: [0]PETSC ERROR: ------------------------------------------------------------------------ also ex12_quad_hpddm_reuse_baij fails with a lot more "Read -1, expected ..." which I don't know where they come from...? Hypre (like in diff-snes_tutorials-ex56_hypre) is also having DIVERGED_INDEFINITE_PC failures... Please see the 3 attached docker files: 1) fedora_mkl_and_devtools : the DockerFile which install fedore 33 with gnu compilers and MKL and everything to develop. 2) openmpi: the DockerFile to bluid OpenMPI 3) petsc: The las DockerFile that build/install and test PETSc I build the 3 like this: docker build -t fedora_mkl_and_devtools -f fedora_mkl_and_devtools . docker build -t openmpi -f openmpi . docker build -t petsc -f petsc . Disclaimer: I am not a docker expert, so I may do things that are not docker-stat-of-the-art but I am opened to suggestions... ;) I have just ran it on my portable (long) which have not enough cores, so many more tests failed (should force --oversubscribe but don't know how to). I will relaunch on my workstation in a few minutes. I will now test your branch! (sorry for the delay). Thanks, Eric On 2021-03-11 9:03 a.m., Eric Chamberland wrote: > > Hi Pierre, > > ok, that's interesting! > > I will try to build a docker image until tomorrow and give you the > exact recipe to reproduce the bugs. > > Eric > > > On 2021-03-11 2:46 a.m., Pierre Jolivet wrote: >> >> >>> On 11 Mar 2021, at 6:16 AM, Barry Smith <bsmith(a)petsc.dev >>> <mailto:[email protected]>> wrote: >>> >>> >>> Eric, >>> >>> Sorry about not being more immediate. We still have this in our >>> active email so you don't need to submit individual issues. We'll >>> try to get to them as soon as we can. >> >> Indeed, I’m still trying to figure this out. >> I realized that some of my configure flags were different than yours, >> e.g., no --with-memalign. >> I’ve also added SuperLU_DIST to my installation. >> Still, I can’t reproduce any issue. >> I will continue looking into this, it appears I’m seeing some >> valgrind errors, but I don’t know if this is some side effect of >> OpenMPI not being valgrind-clean (last time I checked, there was no >> error with MPICH). >> >> Thank you for your patience, >> Pierre >> >> /usr/bin/gmake -f gmakefile test test-fail=1 >> Using MAKEFLAGS: test-fail=1 >> TEST >> arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex12_quad_hpddm_reuse_baij.counts >> ok snes_tutorials-ex12_quad_hpddm_reuse_baij >> ok diff-snes_tutorials-ex12_quad_hpddm_reuse_baij >> TEST >> arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tests-ex33_superlu_dist_2.counts >> ok ksp_ksp_tests-ex33_superlu_dist_2 >> ok diff-ksp_ksp_tests-ex33_superlu_dist_2 >> TEST >> arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tests-ex49_superlu_dist.counts >> ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-0 >> ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-0 >> ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-1 >> ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-0_conv-1 >> ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-0 >> ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-0 >> ok ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-1 >> ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-1herm-1_conv-1 >> ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-0 >> ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-0 >> ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-1 >> ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-0_conv-1 >> ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-0 >> ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-0 >> ok ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-1 >> ok diff-ksp_ksp_tests-ex49_superlu_dist+nsize-4herm-1_conv-1 >> TEST >> arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex50_tut_2.counts >> ok ksp_ksp_tutorials-ex50_tut_2 >> ok diff-ksp_ksp_tutorials-ex50_tut_2 >> TEST >> arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tests-ex33_superlu_dist.counts >> ok ksp_ksp_tests-ex33_superlu_dist >> ok diff-ksp_ksp_tests-ex33_superlu_dist >> TEST >> arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex56_hypre.counts >> ok snes_tutorials-ex56_hypre >> ok diff-snes_tutorials-ex56_hypre >> TEST >> arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex56_2.counts >> ok ksp_ksp_tutorials-ex56_2 >> ok diff-ksp_ksp_tutorials-ex56_2 >> TEST >> arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex17_3d_q3_trig_elas.counts >> ok snes_tutorials-ex17_3d_q3_trig_elas >> ok diff-snes_tutorials-ex17_3d_q3_trig_elas >> TEST >> arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex12_quad_hpddm_reuse_threshold_baij.counts >> ok snes_tutorials-ex12_quad_hpddm_reuse_threshold_baij >> ok diff-snes_tutorials-ex12_quad_hpddm_reuse_threshold_baij >> TEST >> arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5_superlu_dist_3.counts >> not ok ksp_ksp_tutorials-ex5_superlu_dist_3 # Error code: 1 >> #srun: error: Unable to create step for job 1426755: More processors >> requested than permitted >> ok ksp_ksp_tutorials-ex5_superlu_dist_3 # SKIP Command failed so no diff >> TEST >> arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5f_superlu_dist.counts >> ok ksp_ksp_tutorials-ex5f_superlu_dist # SKIP Fortran required for >> this test >> TEST >> arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex12_tri_parmetis_hpddm_baij.counts >> ok snes_tutorials-ex12_tri_parmetis_hpddm_baij >> ok diff-snes_tutorials-ex12_tri_parmetis_hpddm_baij >> TEST >> arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex19_tut_3.counts >> ok snes_tutorials-ex19_tut_3 >> ok diff-snes_tutorials-ex19_tut_3 >> TEST >> arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex17_3d_q3_trig_vlap.counts >> ok snes_tutorials-ex17_3d_q3_trig_vlap >> ok diff-snes_tutorials-ex17_3d_q3_trig_vlap >> TEST >> arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5f_superlu_dist_3.counts >> ok ksp_ksp_tutorials-ex5f_superlu_dist_3 # SKIP Fortran required for >> this test >> TEST >> arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex19_superlu_dist.counts >> ok snes_tutorials-ex19_superlu_dist >> ok diff-snes_tutorials-ex19_superlu_dist >> TEST >> arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex56_attach_mat_nearnullspace-1_bddc_approx_hypre.counts >> ok snes_tutorials-ex56_attach_mat_nearnullspace-1_bddc_approx_hypre >> ok diff-snes_tutorials-ex56_attach_mat_nearnullspace-1_bddc_approx_hypre >> TEST >> arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex49_hypre_nullspace.counts >> ok ksp_ksp_tutorials-ex49_hypre_nullspace >> ok diff-ksp_ksp_tutorials-ex49_hypre_nullspace >> TEST >> arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex19_superlu_dist_2.counts >> ok snes_tutorials-ex19_superlu_dist_2 >> ok diff-snes_tutorials-ex19_superlu_dist_2 >> TEST >> arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5_superlu_dist_2.counts >> not ok ksp_ksp_tutorials-ex5_superlu_dist_2 # Error code: 1 >> #srun: error: Unable to create step for job 1426755: More processors >> requested than permitted >> ok ksp_ksp_tutorials-ex5_superlu_dist_2 # SKIP Command failed so no diff >> TEST >> arch-linux2-c-opt-ompi/tests/counts/snes_tutorials-ex56_attach_mat_nearnullspace-0_bddc_approx_hypre.counts >> ok snes_tutorials-ex56_attach_mat_nearnullspace-0_bddc_approx_hypre >> ok diff-snes_tutorials-ex56_attach_mat_nearnullspace-0_bddc_approx_hypre >> TEST >> arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex64_1.counts >> ok ksp_ksp_tutorials-ex64_1 >> ok diff-ksp_ksp_tutorials-ex64_1 >> TEST >> arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5_superlu_dist.counts >> not ok ksp_ksp_tutorials-ex5_superlu_dist # Error code: 1 >> #srun: error: Unable to create step for job 1426755: More processors >> requested than permitted >> ok ksp_ksp_tutorials-ex5_superlu_dist # SKIP Command failed so no diff >> TEST >> arch-linux2-c-opt-ompi/tests/counts/ksp_ksp_tutorials-ex5f_superlu_dist_2.counts >> ok ksp_ksp_tutorials-ex5f_superlu_dist_2 # SKIP Fortran required for >> this test >> >>> Barry >>> >>> >>>> On Mar 10, 2021, at 11:03 PM, Eric Chamberland >>>> <Eric.Chamberland(a)giref.ulaval.ca >>>> <mailto:[email protected]>> wrote: >>>> >>>> Barry, >>>> >>>> to get a some follow up on --with-openmp=1 failures, shall I open >>>> gitlab issues for: >>>> >>>> a) all hypre failures giving DIVERGED_INDEFINITE_PC >>>> >>>> b) all superlu_dist failures giving different results with initia >>>> and "Exceeded timeout limit of 60 s" >>>> >>>> c) hpddm failures "free(): invalid next size (fast)" and >>>> "Segmentation Violation" >>>> >>>> d) all tao's "Exceeded timeout limit of 60 s" >>>> >>>> I don't see how I could do all these debugging by myself... >>>> >>>> Thanks, >>>> >>>> Eric >>>> >>>> >>> >> > -- > Eric Chamberland, ing., M. Ing > Professionnel de recherche > GIREF/Université Laval > (418) 656-2131 poste 41 22 42 -- Eric Chamberland, ing., M. Ing Professionnel de recherche GIREF/Université Laval (418) 656-2131 poste 41 22 42

1 0

Re: [petsc-dev] Fortran macOS Linkage Warning
by Barry Smith 12 Mar '21

12 Mar '21

You should be able to edit lib/petsc/conf/rules and add it to the filter for .cu.PETSc .kokkos.cxx.PETSc .c.PETSc .cxx.PETSc: and .F.PETSc .F90.PETSc: I think it is fine to hide this warning; it is impossible for me to understand anyway :-(. Barry > On Mar 12, 2021, at 2:44 AM, Pierre Jolivet <pierre(a)joliv.et> wrote: > > Sorry to dig up this old thread, but this is triggering false positives while make checking for people who don’t want to toy around with MACOSX_DEPLOYMENT_TARGET. > Could/should we filter this somehow, or have BuildSystem tinker with the environment variable itself? > > Thanks, > Pierre > > $ make check > Running check examples to verify correct installation > Using PETSC_DIR=/Users/jolivet/repo/petsc and PETSC_ARCH=arch-darwin-c-debug-real > *******************Error detected during compile or link!******************* > See http://www.mcs.anl.gov/petsc/documentation/faq.html <http://www.mcs.anl.gov/petsc/documentation/faq.html> > /Users/jolivet/repo/petsc/src/snes/tutorials ex19 > ********************************************************************************* > mpicc -Wl,-bind_at_load -Wl,-multiply_defined,suppress -Wl,-multiply_defined -Wl,suppress -Wl,-commons,use_dylibs -Wl,-search_paths_first -Wl,-no_compact_unwind -Wall -g -Wall -g -I/Users/jolivet/repo/petsc/include -I/Users/jolivet/repo/petsc/arch-darwin-c-debug-real/include -I/opt/intel/mkl/include ex19.c -Wl,-rpath,/Users/jolivet/repo/petsc/arch-darwin-c-debug-real/lib -L/Users/jolivet/repo/petsc/arch-darwin-c-debug-real/lib -Wl,-rpath,/Users/jolivet/repo/petsc/arch-darwin-c-debug-real/lib -L/Users/jolivet/repo/petsc/arch-darwin-c-debug-real/lib -Wl,-rpath,/opt/intel/mkl/lib -L/opt/intel/mkl/lib -Wl,-rpath,/opt/intel/lib -L/opt/intel/lib -Wl,-rpath,/usr/local/Cellar/mpich/3.4.1_1/lib -L/usr/local/Cellar/mpich/3.4.1_1/lib -Wl,-rpath,/usr/local/Cellar/gcc/10.2.0_4/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0 -L/usr/local/Cellar/gcc/10.2.0_4/lib/gcc/10/gcc/x86_64-apple-darwin20/10.2.0 -Wl,-rpath,/usr/local/Cellar/gcc/10.2.0_4/lib/gcc/10 -L/usr/local/Cellar/gcc/10.2.0_4/lib/gcc/10 -lpetsc -lHYPRE -lcmumps -ldmumps -lsmumps -lzmumps -lmumps_common -lpord -lmkl_scalapack_lp64 -lmkl_blacs_mpich_lp64 -lEl -lElSuiteSparse -lpmrrr -lp4est -lsc -liomp5 -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lparmmg -lmmg -lmmg3d -lptesmumps -lptscotchparmetis -lptscotch -lptscotcherr -lesmumps -lscotch -lscotcherr -lparmetis -lmetis -ltriangle -ltet -lz -lX11 -lc++ -ldl -lmpifort -lmpi -lpmpi -lgfortran -lquadmath -lm -lc++ -ldl -o ex19 > ld: warning: dylib (/usr/local/Cellar/mpich/3.4.1_1/lib/libmpifort.dylib) was built for newer macOS version (11.1) than being linked (11.0) > ld: warning: dylib (/usr/local/Cellar/gcc/10.2.0_4/lib/gcc/10/libgfortran.dylib) was built for newer macOS version (11.2) than being linked (11.0) > ld: warning: dylib (/usr/local/Cellar/gcc/10.2.0_4/lib/gcc/10/libquadmath.dylib) was built for newer macOS version (11.2) than being linked (11.0) > ld: warning: dylib (/usr/local/Cellar/gcc/10.2.0_4/lib/gcc/10/libquadmath.0.dylib) was built for newer macOS version (11.2) than being linked (11.0) > ld: warning: object file (/Users/jolivet/repo/petsc/arch-darwin-c-debug-real/lib/libcmumps.a(cana_aux.o)) was built for newer macOS version (11.2) than being linked (11.0) > ld: warning: object file (/Users/jolivet/repo/petsc/arch-darwin-c-debug-real/lib/libdmumps.a(dana_aux.o)) was built for newer macOS version (11.2) than being linked (11.0) > ld: warning: object file (/Users/jolivet/repo/petsc/arch-darwin-c-debug-real/lib/libsmumps.a(sana_aux.o)) was built for newer macOS version (11.2) than being linked (11.0) > ld: warning: object file (/Users/jolivet/repo/petsc/arch-darwin-c-debug-real/lib/libzmumps.a(zana_aux.o)) was built for newer macOS version (11.2) than being linked (11.0) > ld: warning: object file (/Users/jolivet/repo/petsc/arch-darwin-c-debug-real/lib/libmumps_common.a(front_data_mgt_m.o)) was built for newer macOS version (11.2) than being linked (11.0) > C/C++ example src/snes/tutorials/ex19 run successfully with 1 MPI process > > >> On 15 Feb 2021, at 6:08 PM, Jacob Faibussowitsch <jacob.fai(a)gmail.com <mailto:[email protected]>> wrote: >> >> Yes, this also seems to work on my end. >> >> Best regards, >> >> Jacob Faibussowitsch >> (Jacob Fai - booss - oh - vitch) >> Cell: (312) 694-3391 >> >>> On Feb 15, 2021, at 12:05, Satish Balay <balay(a)mcs.anl.gov <mailto:[email protected]>> wrote: >>> >>> Does setting MACOSX_DEPLOYMENT_TARGET also work? >>> >>> Satish >>> >>> On Mon, 15 Feb 2021, Jacob Faibussowitsch wrote: >>> >>>> I think this problem ultimately stems from the fact that gcc/g++ and gfortran are not from the same people on macOS. It seems like clang is configured to build for 11.0 target, whilst gcc defaults to newest available target. And indeed if I change my compile command there is no warning: >>>> >>>> mkdir -p build && gfortran -c -o ./build/conftest.o conftest.F90 && gcc-10 -o ./build/libconftest.dylib ./build/conftest.o -dynamiclib -undefined dynamic_lookup && rm -rf ./build >>>> >>>> Note I now call gcc-10 as the linker which is the __actual__ gcc (not clang in a trench coat). >>>> >>>> Best regards, >>>> >>>> Jacob Faibussowitsch >>>> (Jacob Fai - booss - oh - vitch) >>>> Cell: (312) 694-3391 >>>> >>>>> On Feb 15, 2021, at 11:46, Jacob Faibussowitsch <jacob.fai(a)gmail.com <mailto:[email protected]>> wrote: >>>>> >>>>> Ok so if I pass -mmacosx-version-min=11.2 to the linker the warning disappears... >>>>> >>>>> Best regards, >>>>> >>>>> Jacob Faibussowitsch >>>>> (Jacob Fai - booss - oh - vitch) >>>>> Cell: (312) 694-3391 >>>>> >>>>>> On Feb 15, 2021, at 11:41, Jacob Faibussowitsch <jacob.fai(a)gmail.com <mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>> wrote: >>>>>> >>>>>> I can reproduce the warning compiling just that simple code you listed. This is the compile command: >>>>>> >>>>>> mkdir -p build && gfortran -c -o ./build/conftest.o conftest.F90 && gcc -o ./build/libconftest.dylib ./build/conftest.o -dynamiclib -undefined dynamic_lookup && rm -rf ./build >>>>>> >>>>>> And error: >>>>>> ld: warning: object file (./build/conftest.o) was built for newer macOS version (11.2) than being linked (11.0) >>>>>> >>>>>> Note I recently uninstalled and reinstalled command line tools too, which does not seem to have fixed it. >>>>>> >>>>>> Best regards, >>>>>> >>>>>> Jacob Faibussowitsch >>>>>> (Jacob Fai - booss - oh - vitch) >>>>>> Cell: (312) 694-3391 >>>>>> >>>>>>> On Feb 15, 2021, at 11:30, Satish Balay <balay(a)mcs.anl.gov <mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>> wrote: >>>>>>> >>>>>>> Here is the compile command and the warning: >>>>>>> >>>>>>>>>>>>> >>>>>>> ===== Checking linker >>>>>>> Executing: gfortran -c -o /var/folders/9w/7dlszmmn6q1gd5yf3r9n79bc0000gn/T/petsc-dk9aiv7x/config.setCompilers/conftest.o -I/var/folders/9w/7dlszmmn6q1gd5yf3r9n79bc0000gn/T/petsc-dk9aiv7x/config.setCompilers /var/folders/9w/7dlszmmn6q1gd5yf3r9n79bc0000gn/T/petsc-dk9aiv7x/config.setCompilers/conftest.F90 >>>>>>> Successful compile: >>>>>>> Source: >>>>>>> function foo(a) >>>>>>> real:: a,x,bar >>>>>>> common /xx/ x >>>>>>> x=a >>>>>>> foo = bar(x) >>>>>>> end >>>>>>> Executing: gcc -o /var/folders/9w/7dlszmmn6q1gd5yf3r9n79bc0000gn/T/petsc-dk9aiv7x/config.setCompilers/libconftest.dylib -dynamiclib -single_module -undefined dynamic_lookup -multiply_defined suppress /var/folders/9w/7dlszmmn6q1gd5yf3r9n79bc0000gn/T/petsc-dk9aiv7x/config.setCompilers/conftest.o >>>>>>> Possible ERROR while running linker: >>>>>>> stderr: >>>>>>> ld: warning: object file (/var/folders/9w/7dlszmmn6q1gd5yf3r9n79bc0000gn/T/petsc-dk9aiv7x/config.setCompilers/conftest.o) was built for newer macOS version (11.2) than being linked (11.0) >>>>>>> Accepted FC PIC code without compiler flag >>>>>>> ============================================================================================= >>>>>>> <<<<<< >>>>>>> >>>>>>> Its likely due to the way brew built gfortran. You can try compiling a simple test code and see if you can reproduce this warning. >>>>>>> >>>>>>> There is a related issue at: >>>>>>> >>>>>>> https://github.com/spack/spack/issues/21644 <https://github.com/spack/spack/issues/21644> <https://github.com/spack/spack/issues/21644 <https://github.com/spack/spack/issues/21644>> >>>>>>> >>>>>>> Satish >>>>>>> >>>>>>> >>>>>>> On Sat, 13 Feb 2021, Jacob Faibussowitsch wrote: >>>>>>> >>>>>>>> Hello All, >>>>>>>> >>>>>>>> I’ve been getting these warnings for weeks, at first I figured it was due to gfortran brew package not being updated for Big Sur, but since it has persisted through a few gfortran brew updates I’m thinking it may be petsc related. This is on clean build on master. >>>>>>>> >>>>>>>> ld: warning: object file (master-arch-darwin-c-debug/obj/sys/classes/bag/f2003-src/fsrc/bagenum.o) was built for newer macOS version (11.2) than being linked (11.0) >>>>>>>> ld: warning: object file (master-arch-darwin-c-debug/obj/sys/f90-src/fsrc/f90_fwrap.o) was built for newer macOS version (11.2) than being linked (11.0) >>>>>>>> ld: warning: object file (master-arch-darwin-c-debug/obj/sys/objects/f2003-src/fsrc/optionenum.o) was built for newer macOS version (11.2) than being linked (11.0) >>>>>>>> ld: warning: object file (master-arch-darwin-c-debug/obj/sys/fsrc/somefort.o) was built for newer macOS version (11.2) than being linked (11.0) >>>>>>>> ld: warning: object file (master-arch-darwin-c-debug/obj/sys/f90-mod/petscsysmod.o) was built for newer macOS version (11.2) than being linked (11.0) >>>>>>>> >>>>>>>> Best regards, >>>>>>>> >>>>>>>> Jacob Faibussowitsch >>>>>>>> (Jacob Fai - booss - oh - vitch) >>>>>>>> Cell: (312) 694-3391 >> >

1 0

Fwd: Argonne GPU Virtual Hackathon - Accepted
by Barry Smith 12 Mar '21

12 Mar '21

Folks, Get was able to get us into the ANL GPU hackathon that will use the "theta" GPU system at ANL (that has NVIDIA GPUs). ALCF accounts are needed for the hackathon (but can be created). Registration information is below. If we go with specific goals it could be a useful experience. Barry > Begin forwarded message: > > From: Andi Moore <amoore(a)nvidia.com> > Subject: Argonne GPU Virtual Hackathon - Accepted > Date: March 11, 2021 at 5:09:02 PM CST > To: "d.getnet(a)gmail.com" <d.getnet(a)gmail.com>, "gbetrie(a)anl.gov" <gbetrie(a)anl.gov>, "bsmith(a)petsc.dev" <bsmith(a)petsc.dev> > Cc: "Ghadar, Yasaman" <ghadar(a)anl.gov>, Julia Levites <jlevites(a)nvidia.com> > Message-Id: <MW3PR12MB4588BF988609B8F7A49F6A21B3909(a)MW3PR12MB4588.namprd12.prod.outlook.com> > > Dear Team Barry Smith, > > Congratulations! Your team has been selected to participate in the Argonne GPU Virtual Hackathon. > > The next steps: > > Registration > Each team member is required to register for the event. Please click here <https://www.gpuhackathons.org/form/gpu-event-registration> to register and select the Hackathon from the drop-down menu. Please register as soon as possible. > > Mentors > Mentors are being selected for each team based on your research fields and coding needs. An introductory call for teams and mentors has been scheduled for Wednesday, April 7th, details below. > > System Access > Request an ALCF Computer User Account <https://accounts.alcf.anl.gov/> if you do not currently have one. If you have an ALCF account, but it is currently inactive, reactivate <https://accounts.alcf.anl.gov/#!/accountReactivate> your account. > Specify the following in the account request/reactivation form: > Project Name: gpu_hack > Principal Investigator (PI): Yasaman Ghadar > > Contact accounts(a)alcf.anl.gov <mailto:[email protected]> if you have questions about your ALCF account. > > Additional information > Review the GPU Hackathon Virtual Attendee Guide <https://docs.google.com/document/d/1M7vacg0GLg38gsJYHOg2tM_M0QArOSCrPiLaQER…> > Review the technical resources website <https://www.gpuhackathons.org/technical-resources> which will help you prepare for the hackathon. The page includes information about profiling tools, compilers, libraries, programming models and more. > > Slack > All communication for the event will occur through the Slack workspace > Please join <https://join.slack.com/t/argonnehack2021/shared_invite/zt-nfio2vwz-f0UeYdXV…> the Slack workspace > > Agenda - All calendar invites are attached > Argonne GPU Hackathon Team and Mentor Meeting: 2:00PM-3:00PM CST – April 7, 2021 Zoom Link <https://zoom.us/j/93527581258?pwd=S01GZVAxS3hLVCsvc0wyR2s2MHJEZz09> > > Argonne GPU Hackathon Day 1: 9:00AM-5:00PM CST – April 20, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09> > > Argonne GPU Hackathon Day 2: 9:00AM-5:00PM CST April 27, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09> > > Argonne GPU Hackathon Day 3: 9:00AM-5:00PM CST April 28, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09> > > Argonne GPU Hackathon Day 4: 9:00AM-5:30PM CST April 29, 2021 Zoom Link <https://zoom.us/j/94645872736?pwd=UmtNakprSjB6WmFVUVdoZUMvdDFIUT09> > > > Please feel free to ask any questions you may have. Thank you all for your interest and outstanding applications! > > Best regards, > The GPU Hackathon Team > >

1 0

Re: [petsc-dev] Are pipelines down?
by Matthew Knepley 12 Mar '21

12 Mar '21

On Fri, Mar 12, 2021 at 10:57 AM Satish Balay <balay(a)mcs.anl.gov> wrote: > And I see you started one pipeline: > > https://gitlab.com/danfinn/petsc/-/jobs/1091965415 > > its started on 'danfinn/petsc' repo - that doesn't have petsc CI > permissions. > > You should 'run pipeline' from > https://gitlab.com/petsc/petsc/-/merge_requests/3707//pipelines > > And the pipeline started by Pierre shows (formatting) errors that need > fixing. > > https://gitlab.com/petsc/petsc/-/jobs/1093419052 Yes, I am fixing those. Thanks, Matt > > Satish > > > On Fri, 12 Mar 2021, Satish Balay via petsc-dev wrote: > > > Only folk in developer group have permissions to use the CI resources > > [to avoid drive by hacks and consumption of resources etc] > > > > And as a developer you have to create a 'new pipeline' - not use the > > auto-started pipeline [this auto-stated pipeline from a fork is from a > > user account that does not have access to CI resources - so these > > pipelines will stay stuck] > > > > I think its best for one of the developers to start the pipeline after > > verifying the code is ok to run. If any of the reviewers approve - > > then I can do this. > > > > Satish > > > > On Fri, 12 Mar 2021, Matthew Knepley wrote: > > > > > On Fri, Mar 12, 2021 at 9:26 AM Pierre Jolivet <pierre(a)joliv.et> > wrote: > > > > > > > You are linking to the forks, i.e., not …/petsc/petsc/…, you’ll > never see > > > > anything going on there, I think. > > > > I’ve started the pipelines by clicking “Run Pipeline” here > > > > https://gitlab.com/petsc/petsc/-/merge_requests/3616/pipelines and > there > > > > https://gitlab.com/petsc/petsc/-/merge_requests/3707/pipelines > > > > > > > > > > Thanks Pierre. There are things I obviously do not understand about > Gitlab > > > :) > > > > > > Matt > > > > > > > > > > Thanks, > > > > Pierre > > > > > > > > On 12 Mar 2021, at 3:02 PM, Matthew Knepley <knepley(a)gmail.com> > wrote: > > > > > > > > On Fri, Mar 12, 2021 at 7:29 AM Matthew Knepley <knepley(a)gmail.com> > wrote: > > > > > > > >> On Thu, Mar 11, 2021 at 11:24 PM Satish Balay <balay(a)mcs.anl.gov> > wrote: > > > >> > > > >>> For forks - a member of the developer group needs to start a new > > > >>> pipeline. (after basic checks on the changes) > > > >>> > > > >> > > > >> The original pipeline just sat pending in the source check, so I > > > >> cancelled it. > > > >> > > > >> I created a new pipeline, but it is just sitting pending in the > source > > > >> check > > > >> > > > >> https://gitlab.com/danfinn/petsc/-/pipelines/269297070 > > > >> > > > >> Why will this not run? > > > >> > > > > > > > > Same problem for Connor's branch: > > > > https://gitlab.com/connorjward/petsc/-/pipelines/269584870 > > > > > > > > Matt > > > > > > > > > > > >> Thanks, > > > >> > > > >> Matt > > > >> > > > >> > > > >>> Satish > > > >>> > > > >>> On Thu, 11 Mar 2021, Jed Brown wrote: > > > >>> > > > >>> > Says the pipeline was canceled. The branch comes from a fork so > it > > > >>> should need approval to run. > > > >>> > > > > >>> > https://gitlab.com/petsc/petsc/-/merge_requests/3707 > > > >>> > > > > >>> > Matthew Knepley <knepley(a)gmail.com> writes: > > > >>> > > > > >>> > > https://gitlab.com/danfinn/petsc/-/pipelines/269297070 > > > >>> > > > > > >>> > > Thanks, > > > >>> > > > > > >>> > > Matt > > > >>> > > > > > >>> > > -- > > > >>> > > What most experimenters take for granted before they begin > their > > > >>> > > experiments is infinitely more interesting than any results to > which > > > >>> their > > > >>> > > experiments lead. > > > >>> > > -- Norbert Wiener > > > >>> > > > > > >>> > > https://www.cse.buffalo.edu/~knepley/ < > > > >>> http://www.cse.buffalo.edu/~knepley/> > > > >>> > > > > >>> > > > >>> > > > >> > > > >> -- > > > >> What most experimenters take for granted before they begin their > > > >> experiments is infinitely more interesting than any results to > which their > > > >> experiments lead. > > > >> -- Norbert Wiener > > > >> > > > >> https://www.cse.buffalo.edu/~knepley/ > > > >> <http://www.cse.buffalo.edu/~knepley/> > > > >> > > > > > > > > > > > > -- > > > > What most experimenters take for granted before they begin their > > > > experiments is infinitely more interesting than any results to which > their > > > > experiments lead. > > > > -- Norbert Wiener > > > > > > > > https://www.cse.buffalo.edu/~knepley/ > > > > <http://www.cse.buffalo.edu/~knepley/> > > > > > > > > > > > > > > > > > > > > > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

1 0