April 2018 - petsc-dev - mailman3.cels.anl.gov

Re: [petsc-dev] MatCreateSubmatrices causes invalid memory write access in fortran
by Hong 18 Apr '18

18 Apr '18

Tim : > Hi all, > > with PETSc v3.9 there seems to be a out of bounds write in the fortran > interface of MatCreateSubmatrices. > https://bitbucket.org/petsc/petsc/src/59479656cffe26dd9a9d446f6d00df > e5e3168168/src/mat/interface/ftn-custom/zmatrixf.c?at= > maint&fileviewer=file-view-default#zmatrixf.c-609 > > The reference mentions that in fortran one should pass a preallocated > array for submat with at least the size of m (actually it's probably a > typo and should be n) > http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/ > MatCreateSubMatrices.html > > Then n+1 entries are copied into that array, where the last entry > seems to be some unneeded dummy (as mentioned). > for (i=0; i<=*n; i++) { /* lsmat[*n] might be a dummy matrix for > saving data struc */ > smat[i] = lsmat[i]; > } > This leads to some bad errors afterwards, when that array is only of > the size of n as recommended. > We need this extra space for reuse -- I agree it makes Fortran API odd. I made following fix to the help manual: diff --git a/src/mat/interface/matrix.c b/src/mat/interface/matrix.c When finished using the submatrices, the user should destroy - them with MatDestroyMatrices(). + them with MatDestroySubMatrices(). Fortran Note: The Fortran interface is slightly different from that given below; it - requires one to pass in as submat a Mat (integer) array of size at least m. + requires one to pass in as submat a Mat (integer) array of size at least n+1. If you agree, I'll push it to the petsc released version. Hong

1 0

Re: [petsc-dev] Matlab interface issue
by Vaclav Hapla 18 Apr '18

18 Apr '18

Barry, yes, it does what is needed. However, I found one more small problem: make check was throwing dyld: Library not loaded: @rpath/libicudata.56.dylib Referenced from: /Users/vhapla/petsc1/src/snes/examples/tutorials/ex19 Reason: image not found I fixed it locally just by adding -Wl,-rpath,/Applications/MATLAB_R2017b.app/bin/maci64 -Wl,-rpath,/Applications/MATLAB_R2017b.app/extern/lib/maci64 to MATLAB_ENGINE_LIB, PETSC_EXTERNAL_LIB_BASIC and PETSC_WITH_EXTERNAL_LIB in $PETSC_ARCH/lib/petsc/conf/petscvariables. Could you please fix configure this way? Thanks Vaclav > 17. 4. 2018 v 23:26, Smith, Barry F. <bsmith(a)mcs.anl.gov>: > > > Please try the branch barry/feature-matlab-socket/maint and --with-matlab-socket=0 and let me know if it works. > > Barry > > >> On Apr 17, 2018, at 3:16 AM, Vaclav Hapla <vaclav.hapla(a)erdw.ethz.ch> wrote: >> >> OK. But definitely it's tricky that you try to inject all PETSc compiler options to the mex wrapper which is by documentation only tested with Xcode and Intel and it's always set up just for one compiler (see mex -setup). >> >> Hence, I would at least vote for config option to turn off this socket mex stuff to bypass potential problems if one doesn't need it. Like --with-matlab-socket=0 or so which would just deactivate the matlabbin target. Or behave like that if --with-matlab-engine=1 and --with-matlab=0 (this currently fails with "Did not find package MATLAB needed by MatlabEngine" which is confusing anyway as it's not a standalone product). >> >> Thanks, >> Vaclav >> >>> 16. 4. 2018 v 20:30, Smith, Barry F. <bsmith(a)mcs.anl.gov>: >>> >>> Hmm, I tried on my Mac and had the same problem. But then I upgraded from Matlab 2017A to 2018A and boom the problem went away and it was able to build the socket mex files without a problem. >>> >>> So my conclusion is that we shouldn't change anything in the PETSc makefiles? >>> >>> Barry >>> >>> >>>> On Apr 16, 2018, at 6:41 AM, Vaclav Hapla <vaclav.hapla(a)erdw.ethz.ch> wrote: >>>> >>>> Hello >>>> >>>> I need to compile PETSc with MATLAB interface on Mac OS. >>>> >>>> First thing is that Mex supports only Xcode or Intel compilers, not GCC. So I compiled PETSc with Xcode, using options >>>> --with-cc="/usr/bin/xcrun -sdk macosx10.13 clang" --with-matlab --with-matlab-arch=maci64 --with-matlab-engine --with-mpi=0 >>>> >>>> But the target "matlabcodes" (in src/sys/classes/viewer/impls/socket/matlab/makefile) was failing with >>>> Undefined symbols for architecture x86_64: >>>> "_main", referenced from: >>>> implicit entry/start for main executable >>>> ld: symbol(s) not found for architecture x86_64 >>>> >>>> After some investigation I found out the problem lies in specifying environment variables to the mex compiler. Mex has some predefined values of these variables and fails if we override them. According to MATLAB documentation, it should be possible to just append what we need, e.g. >>>> LDFLAGS='${LDFLAGS} ${PETSC_EXTERNAL_LIB_BASIC}' >>>> but this apparently does not work as excepted, some predefined options are removed this way. I was debugging that by adding -v to ${MATLAB_MEX} calls in the makefile. It seems the injected LDFLAGS is interpreted _after_ something more is added to LDFLAGS by MEX (stupid). >>>> >>>> What works for me is specifying the options directly do ${MATLAB_MEX}. This is also tricky, though, as ${MATLAB_MEX} does not accept all options, e.g. -Wl,-rpath. So I ended up with replacing >>>> GCC='${CC}' CC='${PCC}' CFLAGS='${COPTFLAGS} ${CC_FLAGS} ${CCPPFLAGS}' LDFLAGS='${PETSC_EXTERNAL_LIB_BASIC}' >>>> by >>>> ${PETSC_CC_INCLUDES} -L${PETSC_DIR}/${PETSC_ARCH}/lib ${PETSC_LIB_BASIC} >>>> in src/sys/classes/viewer/impls/socket/matlab/makefile. See the attached patch. This at least compiled. >>>> >>>> >>>> Do you think this could break anything? Do you have some better idea? >>>> >>>> BTW It would be better if the matlabcodes target could be turned off by configure, if one is interested only in using MATLAB Engine and wants to avoid problems with the mex compiler. >>>> >>>> Thanks, >>>> Vaclav >>>> <patch> >>> >> >

1 0

MatCreateSubmatrices causes invalid memory write access in fortran
by Tim Steinhoff 18 Apr '18

18 Apr '18

Hi all, with PETSc v3.9 there seems to be a out of bounds write in the fortran interface of MatCreateSubmatrices. https://bitbucket.org/petsc/petsc/src/59479656cffe26dd9a9d446f6d00dfe5e3168… The reference mentions that in fortran one should pass a preallocated array for submat with at least the size of m (actually it's probably a typo and should be n) http://www.mcs.anl.gov/petsc/petsc-current/docs/manualpages/Mat/MatCreateSu… Then n+1 entries are copied into that array, where the last entry seems to be some unneeded dummy (as mentioned). for (i=0; i<=*n; i++) { /* lsmat[*n] might be a dummy matrix for saving data struc */ smat[i] = lsmat[i]; } This leads to some bad errors afterwards, when that array is only of the size of n as recommended. Thanks and kind regards, Volker

1 0

*unwind* next branch [again]
by Satish Balay 17 Apr '18

17 Apr '18

All (next users), Next is unwound again so please reset your next before pushing to it. [this is the easiest way to remove branches from next] Old next is saved at next-apr-2018-2 Satish --------------- Ref: https://lists.mcs.anl.gov/pipermail/petsc-dev/2018-April/022747.html So if you have/use 'next' branch in your clone - please do the following (in all your git clones): git checkout master git fetch git branch -D next git checkout next Please *do not push* from a stale 'next' branch in your clone. If this is attempted, the merge step should get a *conflict* on include/petscversion.h. i.e this merge conflict indicates you have to recreate next - and *not* resolve/push. $ git fetch $ git merge origin/next Auto-merging include/petscversion.h CONFLICT (content): Merge conflict in include/petscversion.h Automatic merge failed; fix conflicts and then commit the result.

1 0

Re: [petsc-dev] Is mpich/master:a8a2b30fd21 tested with Petsc?
by Junchao Zhang 17 Apr '18

17 Apr '18

Min, I suggest MPICH add tests to play with the maximal MPI tag (through attribute MPI_TAG_UB). PETSc uses tags from the maximal and downwards. I guess MPICH tests use small tags. That is why the bug only showed up with PETSc. --Junchao Zhang On Tue, Apr 17, 2018 at 3:58 PM, Min Si <msi(a)anl.gov> wrote: > Hi all, > > Thanks for narrowing down the problem. I checked the MPICH code and > believe this is a bug in MPICH. I just created a PR to fix it: > https://github.com/pmodels/mpich/pull/3097 > > It should be merged into MPICH master branch soon. > > Thanks, > Min > > > On 2018/04/17 14:10, Eric Chamberland wrote: > >> Hi, >> >> are we talking about the "tag" passed to MPI_Isend for example? >> >> but does that mean there is something to change for any MPI call which >> involves tags usage or is it only a PETSc "bad" tag usage? >> >> thanks Satish for your finding! >> >> Eric >> >> On 16/04/18 11:31 PM, Satish Balay wrote: >> >>> On Tue, 13 Mar 2018, Eric Chamberland wrote: >>> >>> Hi, >>>> >>>> each night we are testing mpich/master with our petsc-based code. I >>>> don't >>>> know if PETSc team is doing the same thing with mpich/master? (Maybe >>>> it is a >>>> good idea?) >>>> >>>> Everything was fine (except the issue >>>> https://github.com/pmodels/mpich/issues/2892) up to commit 7b8d64debd, >>>> but >>>> since commit mpich:a8a2b30fd21), I have a segfault on a any parallel >>>> nightly >>>> test. >>>> >>> >>> I attempted a bisect of the above range of commits - and narrowed down >>> to: >>> >>> >>>>>>>>>> db11d4c4a70e39a28be88ed32f00542301699e08 is the first bad commit >>> <<<<<<< >>> >>>> >>>>>>>>>>> balay@asterix /home/balay/soft/build/mpich >>> ((db11d4c4a...)|BISECTING) >>> $ git show db11d4c4a70e39a28be88ed32f00542301699e08 >>> commit db11d4c4a70e39a28be88ed32f00542301699e08 (HEAD, refs/bisect/bad) >>> Author: Ken Raffenetti <raffenet(a)mcs.anl.gov> >>> Date: Thu Feb 15 11:37:59 2018 -0600 >>> >>> init: Fix tag upper limit initialization >>> The starting point for this value is equivalent to the usable >>> tag bits >>> macro. This value should be set before device initialization, >>> otherwise devices will assume they have more bits than are actually >>> available. >>> Signed-off-by: Wesley Bland <wesley.bland(a)intel.com> >>> >>> diff --git a/src/mpi/init/initthread.c b/src/mpi/init/initthread.c >>> index cbc41f4d5..b31ae2f07 100644 >>> --- a/src/mpi/init/initthread.c >>> +++ b/src/mpi/init/initthread.c >>> @@ -403,7 +403,7 @@ int MPIR_Init_thread(int *argc, char ***argv, int >>> required, int *provided) >>> MPIR_Process.attrs.host = MPI_PROC_NULL; >>> MPIR_Process.attrs.io = MPI_PROC_NULL; >>> MPIR_Process.attrs.lastusedcode = MPI_ERR_LASTCODE; >>> - MPIR_Process.attrs.tag_ub = 0; >>> + MPIR_Process.attrs.tag_ub = MPIR_TAG_USABLE_BITS; >>> MPIR_Process.attrs.universe = MPIR_UNIVERSE_SIZE_NOT_SET; >>> MPIR_Process.attrs.wtime_is_global = 0; >>> @@ -531,13 +531,6 @@ int MPIR_Init_thread(int *argc, char ***argv, int >>> required, int *provided) >>> MPIR_Assert(((unsigned) MPIR_Process. >>> attrs.tag_ub & ((unsigned) MPIR_Process.attrs.tag_ub >>> + 1)) == 0); >>> - /* Set aside tag space for tagged collectives and failure >>> notification */ >>> -#ifdef HAVE_TAG_ERROR_BITS >>> - MPIR_Process.attrs.tag_ub >>= 3; >>> -#else >>> - MPIR_Process.attrs.tag_ub >>= 1; >>> -#endif >>> - >>> /* Assert: tag_ub is at least the minimum asked for in the MPI >>> spec */ >>> MPIR_Assert(MPIR_Process.attrs.tag_ub >= 32767); >>> <<<<<<<<<<<<<<<<< >>> >>> Reverthing this patch gets mpich-3.3b2 working with petsc >>> >>> Satish >>> >>> >

1 0

Re: [petsc-dev] Is mpich/master:a8a2b30fd21 tested with Petsc?
by Satish Balay 17 Apr '18

17 Apr '18

Thanks! I tried the patch - and this testcase doesn't hang anymore.. Satish On Tue, 17 Apr 2018, Min Si wrote: > Hi all, > > Thanks for narrowing down the problem. I checked the MPICH code and believe > this is a bug in MPICH. I just created a PR to fix it: > https://github.com/pmodels/mpich/pull/3097 > > It should be merged into MPICH master branch soon. > > Thanks, > Min > > On 2018/04/17 14:10, Eric Chamberland wrote: > > Hi, > > > > are we talking about the "tag" passed to MPI_Isend for example? > > > > but does that mean there is something to change for any MPI call which > > involves tags usage or is it only a PETSc "bad" tag usage? > > > > thanks Satish for your finding! > > > > Eric > > > > On 16/04/18 11:31 PM, Satish Balay wrote: > >> On Tue, 13 Mar 2018, Eric Chamberland wrote: > >> > >>> Hi, > >>> > >>> each night we are testing mpich/master with our petsc-based code.ï¿œ I don't > >>> know if PETSc team is doing the same thing with mpich/master?ï¿œï¿œ (Maybe it > >>> is a > >>> good idea?) > >>> > >>> Everything was fine (except the issue > >>> https://github.com/pmodels/mpich/issues/2892) up to commit 7b8d64debd, but > >>> since commit mpich:a8a2b30fd21), I have a segfault on a any parallel > >>> nightly > >>> test. > >> > >> I attempted a bisect of the above range of commits - and narrowed down to: > >> > >>>>>>>>> > >> db11d4c4a70e39a28be88ed32f00542301699e08 is the first bad commit > >> <<<<<<< > >>>>>>>>>> > >> balay@asterix /home/balay/soft/build/mpich ((db11d4c4a...)|BISECTING) > >> $ git show db11d4c4a70e39a28be88ed32f00542301699e08 > >> commit db11d4c4a70e39a28be88ed32f00542301699e08 (HEAD, refs/bisect/bad) > >> Author: Ken Raffenetti <raffenet(a)mcs.anl.gov> > >> Date:ï¿œï¿œ Thu Feb 15 11:37:59 2018 -0600 > >> > >> ï¿œï¿œï¿œï¿œ init: Fix tag upper limit initialization > >> ï¿œï¿œï¿œï¿œ ï¿œï¿œï¿œï¿œ The starting point for this value is equivalent to the usable tag > >> bits > >> ï¿œï¿œï¿œï¿œ macro. This value should be set before device initialization, > >> ï¿œï¿œï¿œï¿œ otherwise devices will assume they have more bits than are actually > >> ï¿œï¿œï¿œï¿œ available. > >> ï¿œï¿œï¿œï¿œ ï¿œï¿œï¿œï¿œ Signed-off-by: Wesley Bland <wesley.bland(a)intel.com> > >> > >> diff --git a/src/mpi/init/initthread.c b/src/mpi/init/initthread.c > >> index cbc41f4d5..b31ae2f07 100644 > >> --- a/src/mpi/init/initthread.c > >> +++ b/src/mpi/init/initthread.c > >> @@ -403,7 +403,7 @@ int MPIR_Init_thread(int *argc, char ***argv, int > >> required, int *provided) > >> ï¿œï¿œï¿œï¿œï¿œ MPIR_Process.attrs.host = MPI_PROC_NULL; > >> ï¿œï¿œï¿œï¿œï¿œ MPIR_Process.attrs.io = MPI_PROC_NULL; > >> ï¿œï¿œï¿œï¿œï¿œ MPIR_Process.attrs.lastusedcode = MPI_ERR_LASTCODE; > >> -ï¿œï¿œï¿œ MPIR_Process.attrs.tag_ub = 0; > >> +ï¿œï¿œï¿œ MPIR_Process.attrs.tag_ub = MPIR_TAG_USABLE_BITS; > >> ï¿œï¿œï¿œï¿œï¿œ MPIR_Process.attrs.universe = MPIR_UNIVERSE_SIZE_NOT_SET; > >> ï¿œï¿œï¿œï¿œï¿œ MPIR_Process.attrs.wtime_is_global = 0; > >> ï¿œ @@ -531,13 +531,6 @@ int MPIR_Init_thread(int *argc, char ***argv, int > >> required, int *provided) > >> ï¿œï¿œï¿œï¿œï¿œ MPIR_Assert(((unsigned) MPIR_Process. > >> ï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œ attrs.tag_ub & ((unsigned) MPIR_Process.attrs.tag_ub + > >> 1)) == 0); > >> ï¿œ -ï¿œï¿œï¿œ /* Set aside tag space for tagged collectives and failure > >> notification */ > >> -#ifdef HAVE_TAG_ERROR_BITS > >> -ï¿œï¿œï¿œ MPIR_Process.attrs.tag_ub >>= 3; > >> -#else > >> -ï¿œï¿œï¿œ MPIR_Process.attrs.tag_ub >>= 1; > >> -#endif > >> - > >> ï¿œï¿œï¿œï¿œï¿œ /* Assert: tag_ub is at least the minimum asked for in the MPI spec > >> */ > >> ï¿œï¿œï¿œï¿œï¿œ MPIR_Assert(MPIR_Process.attrs.tag_ub >= 32767); > >> <<<<<<<<<<<<<<<<< > >> > >> Reverthing this patch gets mpich-3.3b2 working with petsc > >> > >> Satish > >> > > >

1 0

Re: [petsc-dev] Matlab interface issue
by Smith, Barry F. 17 Apr '18

17 Apr '18

Please try the branch barry/feature-matlab-socket/maint and --with-matlab-socket=0 and let me know if it works. Barry > On Apr 17, 2018, at 3:16 AM, Vaclav Hapla <vaclav.hapla(a)erdw.ethz.ch> wrote: > > OK. But definitely it's tricky that you try to inject all PETSc compiler options to the mex wrapper which is by documentation only tested with Xcode and Intel and it's always set up just for one compiler (see mex -setup). > > Hence, I would at least vote for config option to turn off this socket mex stuff to bypass potential problems if one doesn't need it. Like --with-matlab-socket=0 or so which would just deactivate the matlabbin target. Or behave like that if --with-matlab-engine=1 and --with-matlab=0 (this currently fails with "Did not find package MATLAB needed by MatlabEngine" which is confusing anyway as it's not a standalone product). > > Thanks, > Vaclav > >> 16. 4. 2018 v 20:30, Smith, Barry F. <bsmith(a)mcs.anl.gov>: >> >> Hmm, I tried on my Mac and had the same problem. But then I upgraded from Matlab 2017A to 2018A and boom the problem went away and it was able to build the socket mex files without a problem. >> >> So my conclusion is that we shouldn't change anything in the PETSc makefiles? >> >> Barry >> >> >>> On Apr 16, 2018, at 6:41 AM, Vaclav Hapla <vaclav.hapla(a)erdw.ethz.ch> wrote: >>> >>> Hello >>> >>> I need to compile PETSc with MATLAB interface on Mac OS. >>> >>> First thing is that Mex supports only Xcode or Intel compilers, not GCC. So I compiled PETSc with Xcode, using options >>> --with-cc="/usr/bin/xcrun -sdk macosx10.13 clang" --with-matlab --with-matlab-arch=maci64 --with-matlab-engine --with-mpi=0 >>> >>> But the target "matlabcodes" (in src/sys/classes/viewer/impls/socket/matlab/makefile) was failing with >>> Undefined symbols for architecture x86_64: >>> "_main", referenced from: >>> implicit entry/start for main executable >>> ld: symbol(s) not found for architecture x86_64 >>> >>> After some investigation I found out the problem lies in specifying environment variables to the mex compiler. Mex has some predefined values of these variables and fails if we override them. According to MATLAB documentation, it should be possible to just append what we need, e.g. >>> LDFLAGS='${LDFLAGS} ${PETSC_EXTERNAL_LIB_BASIC}' >>> but this apparently does not work as excepted, some predefined options are removed this way. I was debugging that by adding -v to ${MATLAB_MEX} calls in the makefile. It seems the injected LDFLAGS is interpreted _after_ something more is added to LDFLAGS by MEX (stupid). >>> >>> What works for me is specifying the options directly do ${MATLAB_MEX}. This is also tricky, though, as ${MATLAB_MEX} does not accept all options, e.g. -Wl,-rpath. So I ended up with replacing >>> GCC='${CC}' CC='${PCC}' CFLAGS='${COPTFLAGS} ${CC_FLAGS} ${CCPPFLAGS}' LDFLAGS='${PETSC_EXTERNAL_LIB_BASIC}' >>> by >>> ${PETSC_CC_INCLUDES} -L${PETSC_DIR}/${PETSC_ARCH}/lib ${PETSC_LIB_BASIC} >>> in src/sys/classes/viewer/impls/socket/matlab/makefile. See the attached patch. This at least compiled. >>> >>> >>> Do you think this could break anything? Do you have some better idea? >>> >>> BTW It would be better if the matlabcodes target could be turned off by configure, if one is interested only in using MATLAB Engine and wants to avoid problems with the mex compiler. >>> >>> Thanks, >>> Vaclav >>> <patch> >> >

1 0

Re: [petsc-dev] Is mpich/master:a8a2b30fd21 tested with Petsc?
by Min Si 17 Apr '18

17 Apr '18

Hi all, Thanks for narrowing down the problem. I checked the MPICH code and believe this is a bug in MPICH. I just created a PR to fix it: https://github.com/pmodels/mpich/pull/3097 It should be merged into MPICH master branch soon. Thanks, Min On 2018/04/17 14:10, Eric Chamberland wrote: > Hi, > > are we talking about the "tag" passed to MPI_Isend for example? > > but does that mean there is something to change for any MPI call which > involves tags usage or is it only a PETSc "bad" tag usage? > > thanks Satish for your finding! > > Eric > > On 16/04/18 11:31 PM, Satish Balay wrote: >> On Tue, 13 Mar 2018, Eric Chamberland wrote: >> >>> Hi, >>> >>> each night we are testing mpich/master with our petsc-based code.ï¿œ I >>> don't >>> know if PETSc team is doing the same thing with mpich/master?ï¿œï¿œ >>> (Maybe it is a >>> good idea?) >>> >>> Everything was fine (except the issue >>> https://github.com/pmodels/mpich/issues/2892) up to commit >>> 7b8d64debd, but >>> since commit mpich:a8a2b30fd21), I have a segfault on a any parallel >>> nightly >>> test. >> >> I attempted a bisect of the above range of commits - and narrowed >> down to: >> >>>>>>>>> >> db11d4c4a70e39a28be88ed32f00542301699e08 is the first bad commit >> <<<<<<< >>>>>>>>>> >> balay@asterix /home/balay/soft/build/mpich ((db11d4c4a...)|BISECTING) >> $ git show db11d4c4a70e39a28be88ed32f00542301699e08 >> commit db11d4c4a70e39a28be88ed32f00542301699e08 (HEAD, refs/bisect/bad) >> Author: Ken Raffenetti <raffenet(a)mcs.anl.gov> >> Date:ï¿œï¿œ Thu Feb 15 11:37:59 2018 -0600 >> >> ï¿œï¿œï¿œï¿œ init: Fix tag upper limit initialization >> ï¿œï¿œï¿œï¿œ ï¿œï¿œï¿œï¿œ The starting point for this value is equivalent to the >> usable tag bits >> ï¿œï¿œï¿œï¿œ macro. This value should be set before device initialization, >> ï¿œï¿œï¿œï¿œ otherwise devices will assume they have more bits than are actually >> ï¿œï¿œï¿œï¿œ available. >> ï¿œï¿œï¿œï¿œ ï¿œï¿œï¿œï¿œ Signed-off-by: Wesley Bland <wesley.bland(a)intel.com> >> >> diff --git a/src/mpi/init/initthread.c b/src/mpi/init/initthread.c >> index cbc41f4d5..b31ae2f07 100644 >> --- a/src/mpi/init/initthread.c >> +++ b/src/mpi/init/initthread.c >> @@ -403,7 +403,7 @@ int MPIR_Init_thread(int *argc, char ***argv, int >> required, int *provided) >> ï¿œï¿œï¿œï¿œï¿œ MPIR_Process.attrs.host = MPI_PROC_NULL; >> ï¿œï¿œï¿œï¿œï¿œ MPIR_Process.attrs.io = MPI_PROC_NULL; >> ï¿œï¿œï¿œï¿œï¿œ MPIR_Process.attrs.lastusedcode = MPI_ERR_LASTCODE; >> -ï¿œï¿œï¿œ MPIR_Process.attrs.tag_ub = 0; >> +ï¿œï¿œï¿œ MPIR_Process.attrs.tag_ub = MPIR_TAG_USABLE_BITS; >> ï¿œï¿œï¿œï¿œï¿œ MPIR_Process.attrs.universe = MPIR_UNIVERSE_SIZE_NOT_SET; >> ï¿œï¿œï¿œï¿œï¿œ MPIR_Process.attrs.wtime_is_global = 0; >> ï¿œ @@ -531,13 +531,6 @@ int MPIR_Init_thread(int *argc, char ***argv, >> int required, int *provided) >> ï¿œï¿œï¿œï¿œï¿œ MPIR_Assert(((unsigned) MPIR_Process. >> ï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œï¿œ attrs.tag_ub & ((unsigned) >> MPIR_Process.attrs.tag_ub + 1)) == 0); >> ï¿œ -ï¿œï¿œï¿œ /* Set aside tag space for tagged collectives and failure >> notification */ >> -#ifdef HAVE_TAG_ERROR_BITS >> -ï¿œï¿œï¿œ MPIR_Process.attrs.tag_ub >>= 3; >> -#else >> -ï¿œï¿œï¿œ MPIR_Process.attrs.tag_ub >>= 1; >> -#endif >> - >> ï¿œï¿œï¿œï¿œï¿œ /* Assert: tag_ub is at least the minimum asked for in the MPI >> spec */ >> ï¿œï¿œï¿œï¿œï¿œ MPIR_Assert(MPIR_Process.attrs.tag_ub >= 32767); >> <<<<<<<<<<<<<<<<< >> >> Reverthing this patch gets mpich-3.3b2 working with petsc >> >> Satish >>

1 0

Re: [petsc-dev] Is mpich/master:a8a2b30fd21 tested with Petsc?
by Eric Chamberland 17 Apr '18

17 Apr '18

Hi, are we talking about the "tag" passed to MPI_Isend for example? but does that mean there is something to change for any MPI call which involves tags usage or is it only a PETSc "bad" tag usage? thanks Satish for your finding! Eric On 16/04/18 11:31 PM, Satish Balay wrote: > On Tue, 13 Mar 2018, Eric Chamberland wrote: > >> Hi, >> >> each night we are testing mpich/master with our petsc-based code. I don't >> know if PETSc team is doing the same thing with mpich/master? (Maybe it is a >> good idea?) >> >> Everything was fine (except the issue >> https://github.com/pmodels/mpich/issues/2892) up to commit 7b8d64debd, but >> since commit mpich:a8a2b30fd21), I have a segfault on a any parallel nightly >> test. > > I attempted a bisect of the above range of commits - and narrowed down to: > >>>>>>>> > db11d4c4a70e39a28be88ed32f00542301699e08 is the first bad commit > <<<<<<< >>>>>>>>> > balay@asterix /home/balay/soft/build/mpich ((db11d4c4a...)|BISECTING) > $ git show db11d4c4a70e39a28be88ed32f00542301699e08 > commit db11d4c4a70e39a28be88ed32f00542301699e08 (HEAD, refs/bisect/bad) > Author: Ken Raffenetti <raffenet(a)mcs.anl.gov> > Date: Thu Feb 15 11:37:59 2018 -0600 > > init: Fix tag upper limit initialization > > The starting point for this value is equivalent to the usable tag bits > macro. This value should be set before device initialization, > otherwise devices will assume they have more bits than are actually > available. > > Signed-off-by: Wesley Bland <wesley.bland(a)intel.com> > > diff --git a/src/mpi/init/initthread.c b/src/mpi/init/initthread.c > index cbc41f4d5..b31ae2f07 100644 > --- a/src/mpi/init/initthread.c > +++ b/src/mpi/init/initthread.c > @@ -403,7 +403,7 @@ int MPIR_Init_thread(int *argc, char ***argv, int required, int *provided) > MPIR_Process.attrs.host = MPI_PROC_NULL; > MPIR_Process.attrs.io = MPI_PROC_NULL; > MPIR_Process.attrs.lastusedcode = MPI_ERR_LASTCODE; > - MPIR_Process.attrs.tag_ub = 0; > + MPIR_Process.attrs.tag_ub = MPIR_TAG_USABLE_BITS; > MPIR_Process.attrs.universe = MPIR_UNIVERSE_SIZE_NOT_SET; > MPIR_Process.attrs.wtime_is_global = 0; > > @@ -531,13 +531,6 @@ int MPIR_Init_thread(int *argc, char ***argv, int required, int *provided) > MPIR_Assert(((unsigned) MPIR_Process. > attrs.tag_ub & ((unsigned) MPIR_Process.attrs.tag_ub + 1)) == 0); > > - /* Set aside tag space for tagged collectives and failure notification */ > -#ifdef HAVE_TAG_ERROR_BITS > - MPIR_Process.attrs.tag_ub >>= 3; > -#else > - MPIR_Process.attrs.tag_ub >>= 1; > -#endif > - > /* Assert: tag_ub is at least the minimum asked for in the MPI spec */ > MPIR_Assert(MPIR_Process.attrs.tag_ub >= 32767); > > <<<<<<<<<<<<<<<<< > > Reverthing this patch gets mpich-3.3b2 working with petsc > > Satish >

1 0

Re: [petsc-dev] Matlab interface issue
by Vaclav Hapla 17 Apr '18

17 Apr '18

OK. But definitely it's tricky that you try to inject all PETSc compiler options to the mex wrapper which is by documentation only tested with Xcode and Intel and it's always set up just for one compiler (see mex -setup). Hence, I would at least vote for config option to turn off this socket mex stuff to bypass potential problems if one doesn't need it. Like --with-matlab-socket=0 or so which would just deactivate the matlabbin target. Or behave like that if --with-matlab-engine=1 and --with-matlab=0 (this currently fails with "Did not find package MATLAB needed by MatlabEngine" which is confusing anyway as it's not a standalone product). Thanks, Vaclav > 16. 4. 2018 v 20:30, Smith, Barry F. <bsmith(a)mcs.anl.gov>: > > Hmm, I tried on my Mac and had the same problem. But then I upgraded from Matlab 2017A to 2018A and boom the problem went away and it was able to build the socket mex files without a problem. > > So my conclusion is that we shouldn't change anything in the PETSc makefiles? > > Barry > > >> On Apr 16, 2018, at 6:41 AM, Vaclav Hapla <vaclav.hapla(a)erdw.ethz.ch> wrote: >> >> Hello >> >> I need to compile PETSc with MATLAB interface on Mac OS. >> >> First thing is that Mex supports only Xcode or Intel compilers, not GCC. So I compiled PETSc with Xcode, using options >> --with-cc="/usr/bin/xcrun -sdk macosx10.13 clang" --with-matlab --with-matlab-arch=maci64 --with-matlab-engine --with-mpi=0 >> >> But the target "matlabcodes" (in src/sys/classes/viewer/impls/socket/matlab/makefile) was failing with >> Undefined symbols for architecture x86_64: >> "_main", referenced from: >> implicit entry/start for main executable >> ld: symbol(s) not found for architecture x86_64 >> >> After some investigation I found out the problem lies in specifying environment variables to the mex compiler. Mex has some predefined values of these variables and fails if we override them. According to MATLAB documentation, it should be possible to just append what we need, e.g. >> LDFLAGS='${LDFLAGS} ${PETSC_EXTERNAL_LIB_BASIC}' >> but this apparently does not work as excepted, some predefined options are removed this way. I was debugging that by adding -v to ${MATLAB_MEX} calls in the makefile. It seems the injected LDFLAGS is interpreted _after_ something more is added to LDFLAGS by MEX (stupid). >> >> What works for me is specifying the options directly do ${MATLAB_MEX}. This is also tricky, though, as ${MATLAB_MEX} does not accept all options, e.g. -Wl,-rpath. So I ended up with replacing >> GCC='${CC}' CC='${PCC}' CFLAGS='${COPTFLAGS} ${CC_FLAGS} ${CCPPFLAGS}' LDFLAGS='${PETSC_EXTERNAL_LIB_BASIC}' >> by >> ${PETSC_CC_INCLUDES} -L${PETSC_DIR}/${PETSC_ARCH}/lib ${PETSC_LIB_BASIC} >> in src/sys/classes/viewer/impls/socket/matlab/makefile. See the attached patch. This at least compiled. >> >> >> Do you think this could break anything? Do you have some better idea? >> >> BTW It would be better if the matlabcodes target could be turned off by configure, if one is interested only in using MATLAB Engine and wants to avoid problems with the mex compiler. >> >> Thanks, >> Vaclav >> <patch> >

1 0