March 2020 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] Globsearch fails for me when running tests
by Satish Balay 28 Mar '20

28 Mar '20

On Sat, 28 Mar 2020, Jed Brown wrote: > Matthew Knepley <knepley(a)gmail.com> writes: > > >> > IIRC, the $(file ...) function does not work with stock make in macOS. > >> > >> You're right; that is a make 4.0 feature. But developers who need > >> globsearch should have the ability to evade Apple's anti-GNU smear. > >> > > > > Do we really have to make things hard on Mac to use something I use every > > day hundreds of times? > > Are you still using make-3.81? Do you also use an Apple Newton? > > You can still use search and searchin. > > Scott can implement globsearch to call Python that calls make print-test > (listing tests on stdout) and returns the result on stdout. But it > can't pass the list all the tests on the command line. > Configure has been printing this warning for a very long time - but its easily ignored.. =============================================================================== ***** WARNING: You have an older version of Gnu make, it will work, but may not support all the parallel testing options. You can install the latest Gnu make with your package manager, such as brew or macports, or use the --download-make option to get the latest Gnu make ***** =============================================================================== Satish

1 0

Re: [petsc-dev] Globsearch fails for me when running tests
by Jed Brown 28 Mar '20

28 Mar '20

Matthew Knepley <knepley(a)gmail.com> writes: >> > IIRC, the $(file ...) function does not work with stock make in macOS. >> >> You're right; that is a make 4.0 feature. But developers who need >> globsearch should have the ability to evade Apple's anti-GNU smear. >> > > Do we really have to make things hard on Mac to use something I use every > day hundreds of times? Are you still using make-3.81? Do you also use an Apple Newton? You can still use search and searchin. Scott can implement globsearch to call Python that calls make print-test (listing tests on stdout) and returns the result on stdout. But it can't pass the list all the tests on the command line.

1 0

New std CPP in MSVC
by Lisandro Dalcin 28 Mar '20

28 Mar '20

https://devblogs.microsoft.com/cppblog/announcing-full-support-for-a-c-c-co… -- Lisandro Dalcin ============ Research Scientist Extreme Computing Research Center (ECRC) King Abdullah University of Science and Technology (KAUST) http://ecrc.kaust.edu.sa/

1 0

Re: [petsc-dev] Globsearch fails for me when running tests
by Matthew Knepley 28 Mar '20

28 Mar '20

On Fri, Mar 27, 2020 at 4:51 PM Jed Brown <jed(a)jedbrown.org> wrote: > Lisandro Dalcin <dalcinl(a)gmail.com> writes: > > > On Thu, 26 Mar 2020 at 00:48, Jed Brown <jed(a)jedbrown.org> wrote: > > > >> Scott, you can't pass '$(alltesttargets)' on the command line like this. > >> > >> TESTTARGETS := $(shell $(PYTHON) -c"import sys,fnmatch,itertools; > >> m=[fnmatch.filter(sys.argv[2].split(),p) for p in sys.argv[1].split()]; > >> print(' '.join(list(itertools.chain.from_iterable(m))))" '$(globsearch)' > >> '$(alltesttargets)') > >> > >> For this feature, probably put them in an argsfile > >> > >> $(file >$(TESTDIR)/globsearch.args,$(alltesttargets)) > >> > >> > > IIRC, the $(file ...) function does not work with stock make in macOS. > > You're right; that is a make 4.0 feature. But developers who need > globsearch should have the ability to evade Apple's anti-GNU smear. > Do we really have to make things hard on Mac to use something I use every day hundreds of times? Matt -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>

1 0

Re: [petsc-dev] Globsearch fails for me when running tests
by Jed Brown 27 Mar '20

27 Mar '20

Lisandro Dalcin <dalcinl(a)gmail.com> writes: > On Thu, 26 Mar 2020 at 00:48, Jed Brown <jed(a)jedbrown.org> wrote: > >> Scott, you can't pass '$(alltesttargets)' on the command line like this. >> >> TESTTARGETS := $(shell $(PYTHON) -c"import sys,fnmatch,itertools; >> m=[fnmatch.filter(sys.argv[2].split(),p) for p in sys.argv[1].split()]; >> print(' '.join(list(itertools.chain.from_iterable(m))))" '$(globsearch)' >> '$(alltesttargets)') >> >> For this feature, probably put them in an argsfile >> >> $(file >$(TESTDIR)/globsearch.args,$(alltesttargets)) >> >> > IIRC, the $(file ...) function does not work with stock make in macOS. You're right; that is a make 4.0 feature. But developers who need globsearch should have the ability to evade Apple's anti-GNU smear.

1 0

Re: [petsc-dev] Globsearch fails for me when running tests
by Lisandro Dalcin 27 Mar '20

27 Mar '20

On Thu, 26 Mar 2020 at 00:48, Jed Brown <jed(a)jedbrown.org> wrote: > Scott, you can't pass '$(alltesttargets)' on the command line like this. > > TESTTARGETS := $(shell $(PYTHON) -c"import sys,fnmatch,itertools; > m=[fnmatch.filter(sys.argv[2].split(),p) for p in sys.argv[1].split()]; > print(' '.join(list(itertools.chain.from_iterable(m))))" '$(globsearch)' > '$(alltesttargets)') > > For this feature, probably put them in an argsfile > > $(file >$(TESTDIR)/globsearch.args,$(alltesttargets)) > > IIRC, the $(file ...) function does not work with stock make in macOS. -- Lisandro Dalcin ============ Research Scientist Extreme Computing Research Center (ECRC) King Abdullah University of Science and Technology (KAUST) http://ecrc.kaust.edu.sa/

1 0

Re: [petsc-dev] Suggestions for MatProductCreate()
by Lisandro Dalcin 27 Mar '20

27 Mar '20

On Fri, 27 Mar 2020 at 05:09, hong(a)aspiritech.org <hong(a)aspiritech.org> wrote: > Lisandro, > >> As I said in previous emails, I'm not complaining about the API, I said >> it was a welcome adition. All I'm asking is: >> >> 1) Fix the handling of reference counting. This is an implementation >> detail, not really an API change. >> > I'm creating a branch from master and patch it with your suggestions on > reference counting. > OK, thanks! > >> 2) Add a simple new routine to the API, MatProductReset(D) [or maybe >> name it MatProductClear(D) ], that should throw away all of the D->product >> stuff to release memory related to the product algorithm implementation. >> After calling MatProductReset(D), you cannot call anymore >> MatProductNumeric(). >> > Do you mean adding a public function MatProductClear(D) to clear internal > data structure 'product' attached to D? > For example, after destroy A and B, user should call MatProductClear(D), > i.e., > MatDestroy(&A) ; > MatDestroy(&B) ; > MatProductClear(D); // throw away all of the D->product stuff, thus D > becomes a standard matrix > > Exactly. MatProductClear(D) should make D a standard matrix and discard all of the internal product stuff. > I understand that you are concerned about the dangling pointers. However, > most users compute D =A*B, then destroy A, B and D separately in random > order. Do we require user to call MatProductClear(D) after A or B is > destroyed? > No, MatProductClear() should not be required. I would code MatProductClear(D) to be a non-op if D does not have the internal product stuff. Then I would always call MatProductClear(D) in MatDestroy(&D) . Of course, all that assuming that MatProductCreate() makes D take ownership (increment refcount) of A and B (and C if given). This way it does not really matter the order in which users destroy A,B, and D. As I said before, look at PCReset() and PCDestroy(). MatProcuctClear() should work similarly as the Reset() routine. > >>> 3) For MatProduct, I consider >>> MatProductSetFromOptions() is logically equivalent to >>> MatSetFromOptions(), >>> MatProductSymbolic() is logically equivalent to MatSetUp(), >>> and >>> MatProductNumeric() assembles the matproduct D repeatedly when A or B >>> are updated. >>> >>> >> OK, so... your point it that what looks like "setup" code >> in MatProductSetFromOptions() is just some minimal stuff required to >> actually dispatch the SetFromOptions_typeA_typeB routine, right? In that >> case, you are right, no changes/refactors needed here. >> > Yes, MatProductSetFromOptions() only > dispatches SetFromOptions_typeA_typeB_producttype() which handles local > options and polymorphism. The organization of new API is described at the > top of petsc/src/mat/interface/matproduct.c. > > OK, all good with that one. Regards, -- Lisandro Dalcin ============ Research Scientist Extreme Computing Research Center (ECRC) King Abdullah University of Science and Technology (KAUST) http://ecrc.kaust.edu.sa/

1 0

Fwd: Suggestions for MatProductCreate()
by hong＠aspiritech.org 26 Mar '20

26 Mar '20

Lisandro, > As I said in previous emails, I'm not complaining about the API, I said it > was a welcome adition. All I'm asking is: > > 1) Fix the handling of reference counting. This is an implementation > detail, not really an API change. > I'm creating a branch from master and patch it with your suggestions on reference counting. > > 2) Add a simple new routine to the API, MatProductReset(D) [or maybe name > it MatProductClear(D) ], that should throw away all of the D->product stuff > to release memory related to the product algorithm implementation. After > calling MatProductReset(D), you cannot call anymore MatProductNumeric(). > Do you mean adding a public function MatProductClear(D) to clear internal data structure 'product' attached to D? For example, after destroy A and B, user should call MatProductClear(D), i.e., MatDestroy(&A) ; MatDestroy(&B) ; MatProductClear(D); // throw away all of the D->product stuff, thus D becomes a standard matrix I understand that you are concerned about the dangling pointers. However, most users compute D =A*B, then destroy A, B and D separately in random order. Do we require user to call MatProductClear(D) after A or B is destroyed? > >> 3) For MatProduct, I consider >> MatProductSetFromOptions() is logically equivalent to >> MatSetFromOptions(), >> MatProductSymbolic() is logically equivalent to MatSetUp(), >> and >> MatProductNumeric() assembles the matproduct D repeatedly when A or B >> are updated. >> >> > OK, so... your point it that what looks like "setup" code > in MatProductSetFromOptions() is just some minimal stuff required to > actually dispatch the SetFromOptions_typeA_typeB routine, right? In that > case, you are right, no changes/refactors needed here. > Yes, MatProductSetFromOptions() only dispatches SetFromOptions_typeA_typeB_producttype() which handles local options and polymorphism. The organization of new API is described at the top of petsc/src/mat/interface/matproduct.c. Hong Note: I sent this reply several times. Due to a problem with my anl email, this email seems not being sent.

1 0

Re: [petsc-dev] petsc release plan for march/2020
by Satish Balay 26 Mar '20

26 Mar '20

All, Please check the changes file - and update any changes/new-features that are missing from here. https://gitlab.com/petsc/petsc/-/blob/master/src/docs/website/documentation… Thanks, Satish On Wed, 25 Mar 2020, Satish Balay via petsc-dev wrote: > > > - feature freeze: March 27 say 5PM EST > > A reminder, feature freeze is in 2 days. I see the following MRs with milestone: v3.13-release > > !2598 Add Sphinx docs and port Developers Manual > !2614 Knepley/feature dm remove hybrid > !2626 More support for not needing to set PETSC_DIR yet build with PETSc, also... > !2634 WIP: Balay/release 3.13 revert to dev > !2554 Knepley/feature fe lagrange gll > !2367 WIP: Add sample file that demonstrates use of CMake and pkg-config to build a PETSc application > > I'm hoping all v3.13-release MRs will complete testing and reviews (approvals) before the freeze. > > Thanks, > Satish > > On Fri, 20 Mar 2020, Satish Balay via petsc-dev wrote: > > > An update: > > > > https://gitlab.com/petsc/petsc/-/merge_requests?milestone_title=v3.13-relea… > > > > We have a week to go. And have about 47 MRs, with 12 marked as 'v3.13-release' milestone. > > > > Likely the milestone is not correctly set for some MRs - if so - please update the MR. > > > > And I see some MRs with significant code changes. Hopefully we can process them early [and not have large changes close to freeze date]. > > > > Satish > > > > On Fri, 13 Mar 2020, Satish Balay via petsc-dev wrote: > > > > > All, > > > > > > We are to make a petsc release by the end of March. > > > > > > For this release [3.13], will work with the following dates: > > > > > > - feature freeze: March 27 say 5PM EST > > > - release: March 29 > > > > > > Merges after freeze should contain only fixes that would normally be acceptable to maint workflow. > > > > > > I've changed the current milestone 'v3.13' to 'master' and created a new one 'v3.13-release' > > > > > > If you are working on a MR with the goal of merging before release - its best to use 'v3.13-release' tag with the MR. > > > > > > And it would be good to avoid merging large changes at the last minute. And not have merge requests stuck in need of reviews, testing and other necessary tasks. > > > > > > And I would think the testing/CI resources would get stressed in this timeframe - so it would be good to use them judiciously if possible. > > > > > > - if there are failures in stage-2 or 3 - and its no longer necessary to complete all the jobs - one can 'cancel' the pipeline. > > > - if a fix needs to be tested - one can first test with only the failed jobs (if this is known) - before doing a full test pipeline. i.e: > > > - start pipeline > > > - immediately cancel the pipeline > > > - now toggle only the jobs that need to be run > > > - [on success of the selected jobs] if one wants to run the full pipeleine - click 'retry' - and the remaining canceled jobs should now get scheduled. > > > > > > thanks, > > > Satish > > > > > >

1 0

Re: [petsc-dev] Very Useful Description of MAT Types in Petsc Index
by Dave May 26 Mar '20

26 Mar '20

On Thu 26. Mar 2020 at 21:19, Jacob Faibussowitsch <jacob.fai(a)gmail.com> wrote: > Actually, the man pages for specific constructors actually yields more > useful information (in nearly all cases) about the formats. > Take a look at these pages > > > https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateS… > > > https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateM… > > > https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateM… > > > https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateM… > > Is that more informative? > > > Yes this is pretty much what I had in mind! > Cool. But I think it needs to moved away from the create calls and to the all > upper-case impls sections. > Totally agree. Just wanted to pointed some useful info is in fact there - albeit in a slightly non obvious place. It’s definitely a better situation than no useful info at all and instead just the same old generic description cut-and-pasted all over the place :D Currently the work-flow of the mat examples I have seen are as follows: > > MatCreate(); > MatSetSizes(); > MatSetType(); > > Which means new users will likely never look at the individual > MatCreateXXX() routines and never see any of the useful information. They > are far more likely to click on the links on the MatSetType calls to the > various implementations, and be frustrated when there’s nothing useful > there... > Sure - I agree with you. > Also as Junchao noted: > > Even better, with examples showing a small matrix. > > > I think this should be on every informative MAT page (wherever that ends > up being). For example for the “base” MatCreateAIJ you have nice diagrams > showing in broad strokes how things are internally structured. Say a user > eventually implements MATMPIAIJMKL, given the above it isn’t guaranteed > that they see the MatCreateAIJ which actually has the useful diagram, > leading to headaches trying to figure out how to properly feed petsc the > matrix. > > It’s certainly a bit of bloat, but every MAT impls should have a section > with the diagram about its “base” implementation. Maybe we can be smart > about it and “link” the text from another part of the src, but I don’t know > much about how much you can do with the current documentation generation. > > Best regards, > > Jacob Faibussowitsch > (Jacob Fai - booss - oh - vitch) > Cell: (312) 694-3391 > > On Mar 26, 2020, at 3:00 PM, Dave May <dave.mayhem23(a)gmail.com> wrote: > > > > On Thu, 26 Mar 2020 at 19:26, Jacob Faibussowitsch <jacob.fai(a)gmail.com> > wrote: > >> Hello, >> >> In keeping with PETSc design it would be nice to have *more* detail for >> all MAT implementations explaining in what the letters stand for, and >> use-cases it might be useful for. For example: >> >> MATAIJSELL >> <https://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Mat/MATAIJSELL.htm…> >> = "AIJSELL" - A matrix type to be used for sparse matrices. This matrix >> type is identical to MATSEQAIJSELL when constructed with a single process >> communicator, and MATMPIAIJSELL otherwise. As a result, for single process >> communicators, MatSeqAIJSetPreallocation >> <https://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Mat/MatSeqAIJSetPr…>() >> is supported, and similarly MatMPIAIJSetPreallocation >> <https://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Mat/MatMPIAIJSetPr…>() >> is supported for communicators controlling multiple processes. It is >> recommended that you call both of the above preallocation routines for >> simplicity. >> >> >> > Agreed that is not particularly informative. > > >> I am no complete beginner but I am no computational matrix expert either. >> I have no idea what “SELL” is. Obviously googling "mat sell format” gives >> less than useful results. Other such types are MATAIJCRL, MATAIJMKL, >> MATAIJPERM, MATADJ. >> > > Actually, the man pages for specific constructors actually yields more > useful information (in nearly all cases) about the formats. > Take a look at these pages > > > https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateS… > > > https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateM… > > > https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateM… > > > https://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateM… > > Is that more informative? > > Admittedly the SELL man page should include references to the papers which > introduced the sliced ELLPACK stuff. > Possibly there are other matrix man pages which could also benefit from > references. > Similarly the source code for these matrix types should ideally also use > the function PetscCitationsRegister() to log / report these papers. > > > >> >> Smallest change could be as simple as mirroring MATSBAIJ, which is clear >> and concise: >> >> MATMPISBAIJ >> <https://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Mat/MATMPISBAIJ.ht…> >> = "mpisbaij" - A matrix type to be used for distributed symmetric sparse >> block matrices, based on block compressed sparse row format. Only the upper >> triangular portion of the "diagonal" portion of the matrix is stored. For >> complex numbers by default this matrix is symmetric, NOT Hermitian >> symmetric. To make it Hermitian symmetric you can call MatSetOption >> <https://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Mat/MatSetOption.h…> >> (Mat >> <https://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Mat/Mat.html#Mat>, >> MAT_HERMITIAN >> <https://www.mcs.anl.gov/petsc/petsc-dev/docs/manualpages/Mat/MatOption.html…> >> ); >> >> >> Clearest case would be: >> >> MATMPISBAIJ = "mpisbaij" - *MATrix MPI Symmetric Block AIJ.* A matrix >> type to be used for distributed symmetric sparse block matrices, based on >> block compressed sparse row format. >> >> Only the upper triangular portion of the "diagonal" portion of the matrix >> is stored. For complex numbers by default this matrix is symmetric, NOT >> Hermitian symmetric. To make it Hermitian symmetric you >> can call MatSetOption(Mat, MAT_HERMITIAN); >> >> >> Each format is useful in its own case, but people can’t use things if >> they don’t know what it is! >> >> Best regards, >> >> Jacob Faibussowitsch >> (Jacob Fai - booss - oh - vitch) >> Cell: (312) 694-3391 >> >> >

1 0