On Tue, 16 Mar 2021, Barry Smith wrote:
Jacob,
You very well may be right, splitting may not be needed or relevant. I only advocated it since for Mat it did fix a VM gfortran memory issue. How closely the IBM compiler and gfortran issues are I do not know.
My MR Mat split is read to to go into main.
https://gitlab.com/petsc/petsc/-/merge_requests/3715 <https://gitlab.com/petsc/petsc/-/merge_requests/3715> The MPI one is still needing some babysitting, hopefully it can pass the CI and get into main soon.
Both should be in main before starting to muck with changing the use stuff for the IBM compiler.
Satish, does your Vec file split pass the CI? If so I think we should also get it into main before messing with the use for the IBM compiler.
Barry - if the switch from 'use' to 'import' is sufficient [for VM builds] - should we still split the sources? Right now it changes only the auto-fortran stubs, fixing custom stubs should give further improvements.
If spliting sources is useful - I'll look into pushing the Vec split into this MR.
BTW: Matt - can you test balay/reorg-f90-for-xlf branch against 'gfortran VM'?
Satish
Who has access to the wonky IBM compiler? Is there any way to have general access so it can be added to the CI and we can maintain code that works with it? Or does someone need to fix things one at a time, first Vec, then Mat, then ....
Barry
On Mar 16, 2021, at 2:14 PM, Satish Balay via petsc-dev <petsc-dev@mcs.anl.gov> wrote:
On Tue, 16 Mar 2021, Matthew Knepley wrote:
On Tue, Mar 16, 2021 at 2:42 PM Jacob Faibussowitsch <jacob.fai@gmail.com>
wrote:
To me the best solution is to first split the files when possible (it is
often not or just too painful) and then adding imports if needed.
I’m not sure I agree. Bear in mind my familiarity with fortran is very
limited, but to me the “use” statement is similar to #include <header> in C
or using namespace xxx in cpp.
Our fortran interface is not optimal code atm, it’s as if we put a
"#include <petscvec.h>” before every Vec function in the C source
(assuming include guards are not used). Sure you can fix this by splitting
the vec src files in two, but at the end of the day that’s a bandaid
solution. I think I can get the import stuff to work reasonably well (a lot
of the infrastructure to capture types is already present in the current
generatefortranstubs).
If you can get this to work, we should definitely do it. Was there a
question about getting it to work?
Likely these additional changes should be done starting with Barry's split-up of sources.
Satish
Thanks,
Matt
Best regards,
Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)
Cell: (312) 694-3391
On Mar 16, 2021, at 12:56, Barry Smith <bsmith@petsc.dev> wrote:
Jacob,
I split the mat definitions in the MR
https://gitlab.com/petsc/petsc/-/merge_requests/3696 and this reduced
memory requirements enough to get builds through on some VM that failed
previously ran out of memory (with gfortran).
The petsc*mod.F90 files are all handwritten so it is ok to manually
change them with imports if that helps the IBM compiler. To me the best
solution is to first split the files when possible (it is often not or just
too painful) and then adding imports if needed.
Note also the MR
https://gitlab.com/petsc/petsc/-/merge_requests/3715/diffs which may help
a great deal with the IBM compiler or not.
Thanks
Barry
On Mar 16, 2021, at 11:47 AM, Jacob Faibussowitsch <jacob.fai@gmail.com>
wrote:
Ok something I have gotten to work, but only doing things by hand in
petscvecmod.F90:
diff --git a/src/vec/f90-mod/petscvecmod.F90
b/src/vec/f90-mod/petscvecmod.F90
index 0c447156b9..ef3e2e2e55 100644
--- a/src/vec/f90-mod/petscvecmod.F90
+++ b/src/vec/f90-mod/petscvecmod.F90
module petscisdef
use petscisdefdummy
interface operator(.ne.)
function isnotequal(A,B)
- use petscisdefdummy
+ import tIS
logical isnotequal
type(tIS), intent(in) :: A,B
end function isnotequal
This works for everything wrapped in a module which contains an interface
block. For dangling functions the following works:
function isnotequal(A,B)
- use petscisdefdummy
+ use petscisdef, only: tIs
logical isnotequal
type(tIS), intent(in) :: A,B
isnotequal = (A%v .ne. B%v)
end function isnotequal
Do our fortran stub generators have any notion of types? Specifically
types that originate from petsc?
Best regards,
Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)
Cell: (312) 694-3391
On Mar 16, 2021, at 11:11, Satish Balay <balay@mcs.anl.gov> wrote:
On Tue, 16 Mar 2021, Jacob Faibussowitsch wrote:
My [partial] change is in branch balay/reorg-f90-for-xlf
Satish is this branch pushed? I’d like to send it to the ibm folks to see
if it works for them as well.
Sorry - pushed now. From what I remember - it didn't work.
Satish
They also added this extra follow up:
The change we did in the source files is to replace all the "use pet*"
statements in all the Interface blocks with IMPORT statement.
The nature of this workaround is to reduce the number of symbols that the
compiler have to create, which exceeded the limitation and caused ICE.
Every USE statement in an interface block opens up the module symbol file
and reads all the symbols from it and creates an entity for each symbol in
compiler. This is unnecessary when the module already has the same USE
statement in the module scope. Instead, users can use IMPORT statement to
make the module symbols accessible inside interface face blocks.
With the change, the compiler would only read the module symbol file once
and create one set of symbols where the old code reads the module symbol
files as many times as the number of the USE statements in Interface blocks
and create that many sets of duplicate symbols. Replacing those USE
statements with IMPORT statements also shortens the compile time
significantly.
Best regards,
Jacob Faibussowitsch
(Jacob Fai - booss - oh - vitch)
Cell: (312) 694-3391
On Mar 3, 2021, at 13:43, Satish Balay via petsc-dev <
petsc-dev@mcs.anl.gov> wrote:
The only change I can get working (i.e avoid compile errors) is to split
petscvecmod.F90 into 2 source files - but I don't know if this will help
with xlf..
My [partial] change is in branch balay/reorg-f90-for-xlf
Satish
On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:
On Wed, 3 Mar 2021, Satish Balay via petsc-dev wrote:
Sure - once any change works locally [for gcc and xlf]
When I try - I get a bunch of errors.. [yet to digest them.]
Can you please give the following source code workaround a try?
Since there is already "use petscvecdefdummy" at the module scope, one
workaround might be to remove the unnecessary "use petscvecdefdummy" in
vecnotequal and vecequals
and all similar procedures.
For example, the test case has:
module petscvecdef
use petscvecdefdummy
...
function vecnotequal(A,B)
use petscvecdefdummy
logical vecnotequal
type(tVec), intent(in) :: A,B
vecnotequal = (A%v .ne. B%v)
end function
Ok - try this suggestion:
diff --git a/src/vec/f90-mod/petscvecmod.F90
b/src/vec/f90-mod/petscvecmod.F90
index 0c447156b9..81968c7ca1 100644
--- a/src/vec/f90-mod/petscvecmod.F90
+++ b/src/vec/f90-mod/petscvecmod.F90
@@ -77,7 +77,6 @@
use petscvecdefdummy
interface operator(.ne.)
function vecnotequal(A,B)
- use petscvecdefdummy
logical vecnotequal
type(tVec), intent(in) :: A,B
end function
FC arch-linux-c-debug/obj/vec/f90-mod/petscvecmod.o
/home/balay/petsc/src/vec/f90-mod/petscvecmod.F90:81:22:
81 | type(tVec), intent(in) :: A,B
| 1
Error: Derived type ‘tvec’ at (1) is being used before it is defined
/home/balay/petsc/include/../src/vec/f90-mod/ftn-auto-interfaces/petscis.h90:2:10:
2 | use petscvecdef
| 1
Fatal Error: Cannot open module file ‘petscvecdef.mod’ for reading at (1):
No such file or directory
<<<<<<
Satish