Confusion/failures about the tests involved in including Hypre
Hello, I'm working on getting a petsc build running on windows. One necessary package to include is Hypre. I've been able to build Hypre seperately using cmake, and confirmed that the library works by setting up a VS project to run some of the example programs. My attempted petsc build is being done through cygwin. I've been able to (with varying degrees of difficulty), build a fairly plain petsc, and one that downloads and builds ptscotch (after some modifications to both ptscotch and the config script). I am now attempting to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc). Note that the same compilers are being used for both Hypre and for petsc through cygwin - the new intel oneapi compilers (icx and ifx, after again varying amounts of pain to work around their awkwardness with the config script). I'm seeing a problem when the config script does some tests on the included hypre lib. The source code looks like: #include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */ #include "HYPRE.h" char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); } int main() { _check_HYPRE_IJMatrixCreate();; return 0; } As I understand this is a fairly standard type of stub program used by the config script to check that it is able to link to certain symbols in given libraries. Tests like this have succeeded in my builds that include PTScotch. I keep getting a linker error with the above test, including if I seperate it out and try to build it seperately: unresolved external symbol "char __cdel HYPRE_IJMatrixCreate(void)" .... Ok, it looks like a problem with either the library or linker commands. But here's the interesting thing - If I transplant this code into VS, with the same project setting that allows it to build the much more nontrivial Hypre example programs, I get the same error: Error LNK2001 unresolved external symbol "char __cdecl HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1 C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1 So it seems like there might be something about this type of stub program that is not working with my Hypre library. I don't fully understand this program - it's able to call the function with no arguments, but it also needs to be linked against a library containing the function, apparently by wrapping it in a static void function? Not something I've seen before. Does anyone have any insight into what might be going wrong - or really just any explaination of how the stub program works so I can figure out why it isn't in this case? Many thanks, Daniel
You don't indicate what type of libraries you built hypre with; static or shared. My guess is you ended up with shared I think the answer to your difficulty is hidden in __cdecl (Satish will know much better than me). When you are looking for symbols in Windows shared libraries you have to prepend something to the function prototype to have it successfully found. For example the PETSc include files have these things __declspec(dllimport) The configure test fails because it does not provide the needed prototype. Likely you built PTScotch with static libraries so no problem. The simplest fix would be to build static hypre libraries. I think it is a major project to get PETSc configure and macro system to work properly with external packages that are in Windows shared libraries since more use of __declspec would be needed. Barry The PETSc installation instructions should probably say something about external packages with Windows shared libraries.
On Jul 19, 2023, at 10:52 AM, Daniel Stone <[email protected]> wrote:
Hello,
I'm working on getting a petsc build running on windows. One necessary package to include is Hypre. I've been able to build Hypre seperately using cmake, and confirmed that the library works by setting up a VS project to run some of the example programs.
My attempted petsc build is being done through cygwin. I've been able to (with varying degrees of difficulty), build a fairly plain petsc, and one that downloads and builds ptscotch (after some modifications to both ptscotch and the config script). I am now attempting to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc). Note that the same compilers are being used for both Hypre and for petsc through cygwin - the new intel oneapi compilers (icx and ifx, after again varying amounts of pain to work around their awkwardness with the config script).
I'm seeing a problem when the config script does some tests on the included hypre lib. The source code looks like:
#include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */
#include "HYPRE.h"
char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main() { _check_HYPRE_IJMatrixCreate();; return 0; }
As I understand this is a fairly standard type of stub program used by the config script to check that it is able to link to certain symbols in given libraries. Tests like this have succeeded in my builds that include PTScotch.
I keep getting a linker error with the above test, including if I seperate it out and try to build it seperately:
unresolved external symbol "char __cdel HYPRE_IJMatrixCreate(void)" ....
Ok, it looks like a problem with either the library or linker commands. But here's the interesting thing - If I transplant this code into VS, with the same project setting that allows it to build the much more nontrivial Hypre example programs, I get the same error:
Error LNK2001 unresolved external symbol "char __cdecl HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1 C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1
So it seems like there might be something about this type of stub program that is not working with my Hypre library. I don't fully understand this program - it's able to call the function with no arguments, but it also needs to be linked against a library containing the function, apparently by wrapping it in a static void function? Not something I've seen before.
Does anyone have any insight into what might be going wrong - or really just any explaination of how the stub program works so I can figure out why it isn't in this case?
Many thanks,
Daniel
You could try skipping this test [and assume --with-hypre-include and --with-hypre-lib options are correct] - and see if this works. diff --git a/config/BuildSystem/config/packages/hypre.py b/config/BuildSystem/config/packages/hypre.py index 5bc88322aa2..2d6c7932e17 100644 --- a/config/BuildSystem/config/packages/hypre.py +++ b/config/BuildSystem/config/packages/hypre.py @@ -11,7 +11,7 @@ class Configure(config.package.GNUPackage): self.requiresversion = 1 self.gitcommit = 'v'+self.version self.download = ['git://https://github.com/hypre-space/hypre','https://github.com/hypre-space/hypre/archive/'+self.gitcommit+'.tar.gz'] - self.functions = ['HYPRE_IJMatrixCreate'] + self.functions = [] self.includes = ['HYPRE.h'] self.liblist = [['libHYPRE.a']] self.buildLanguages = ['C','Cxx'] Satish On Wed, 19 Jul 2023, Barry Smith wrote:
You don't indicate what type of libraries you built hypre with; static or shared. My guess is you ended up with shared
I think the answer to your difficulty is hidden in __cdecl (Satish will know much better than me). When you are looking for symbols in Windows shared libraries you have to prepend something to the function prototype to have it successfully found. For example the PETSc include files have these things __declspec(dllimport) The configure test fails because it does not provide the needed prototype. Likely you built PTScotch with static libraries so no problem.
The simplest fix would be to build static hypre libraries. I think it is a major project to get PETSc configure and macro system to work properly with external packages that are in Windows shared libraries since more use of __declspec would be needed.
Barry
The PETSc installation instructions should probably say something about external packages with Windows shared libraries.
On Jul 19, 2023, at 10:52 AM, Daniel Stone <[email protected]> wrote:
Hello,
I'm working on getting a petsc build running on windows. One necessary package to include is Hypre. I've been able to build Hypre seperately using cmake, and confirmed that the library works by setting up a VS project to run some of the example programs.
My attempted petsc build is being done through cygwin. I've been able to (with varying degrees of difficulty), build a fairly plain petsc, and one that downloads and builds ptscotch (after some modifications to both ptscotch and the config script). I am now attempting to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc). Note that the same compilers are being used for both Hypre and for petsc through cygwin - the new intel oneapi compilers (icx and ifx, after again varying amounts of pain to work around their awkwardness with the config script).
I'm seeing a problem when the config script does some tests on the included hypre lib. The source code looks like:
#include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */
#include "HYPRE.h"
char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main() { _check_HYPRE_IJMatrixCreate();; return 0; }
As I understand this is a fairly standard type of stub program used by the config script to check that it is able to link to certain symbols in given libraries. Tests like this have succeeded in my builds that include PTScotch.
I keep getting a linker error with the above test, including if I seperate it out and try to build it seperately:
unresolved external symbol "char __cdel HYPRE_IJMatrixCreate(void)" ....
Ok, it looks like a problem with either the library or linker commands. But here's the interesting thing - If I transplant this code into VS, with the same project setting that allows it to build the much more nontrivial Hypre example programs, I get the same error:
Error LNK2001 unresolved external symbol "char __cdecl HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1 C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1
So it seems like there might be something about this type of stub program that is not working with my Hypre library. I don't fully understand this program - it's able to call the function with no arguments, but it also needs to be linked against a library containing the function, apparently by wrapping it in a static void function? Not something I've seen before.
Does anyone have any insight into what might be going wrong - or really just any explaination of how the stub program works so I can figure out why it isn't in this case?
Many thanks,
Daniel
BTW: Some explanation of configure: It attempts the following on linux:
Source: #include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */ char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } <<<<<<< Note - it does not include 'HYPRE.h' here - but redefines the prototype as 'char HYPRE_IJMatrixCreate(); Compiling it manually:
[balay@pj01 petsc]$ cat conftest.c char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } [balay@pj01 petsc]$ gcc -c conftest.c [balay@pj01 petsc]$ nm -Ao conftest.o |grep HYPRE_IJMatrixCreate conftest.o:0000000000000000 t _check_HYPRE_IJMatrixCreate conftest.o: U HYPRE_IJMatrixCreate [balay@pj01 petsc]$ nm -Ao arch-linux-c-debug/lib/libHYPRE.so |grep HYPRE_IJMatrixCreate arch-linux-c-debug/lib/libHYPRE.so:000000000007f2c2 T HYPRE_IJMatrixCreate [balay@pj01 petsc]$ <<<< Here the "U HYPRE_IJMatrixCreate" in conftest.o matches "T HYPRE_IJMatrixCreate" in libHYPRE.so - so the "link" test in configure succeeds!
[balay@pj01 petsc]$ gcc -o conftest conftest.o arch-linux-c-debug/lib/libHYPRE.so [balay@pj01 petsc]$ echo $? 0 <<<<<
On windows - [due to name mangling by cdecl/stdcall, (/MT vs /MD) etc..] - this might not match - resulting in link failures. Satish On Wed, 19 Jul 2023, Satish Balay via petsc-users wrote:
You could try skipping this test [and assume --with-hypre-include and --with-hypre-lib options are correct] - and see if this works.
diff --git a/config/BuildSystem/config/packages/hypre.py b/config/BuildSystem/config/packages/hypre.py index 5bc88322aa2..2d6c7932e17 100644 --- a/config/BuildSystem/config/packages/hypre.py +++ b/config/BuildSystem/config/packages/hypre.py @@ -11,7 +11,7 @@ class Configure(config.package.GNUPackage): self.requiresversion = 1 self.gitcommit = 'v'+self.version self.download = ['git://https://github.com/hypre-space/hypre','https://github.com/hypre-space/hypre/archive/'+self.gitcommit+'.tar.gz'] - self.functions = ['HYPRE_IJMatrixCreate'] + self.functions = [] self.includes = ['HYPRE.h'] self.liblist = [['libHYPRE.a']] self.buildLanguages = ['C','Cxx']
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
You don't indicate what type of libraries you built hypre with; static or shared. My guess is you ended up with shared
I think the answer to your difficulty is hidden in __cdecl (Satish will know much better than me). When you are looking for symbols in Windows shared libraries you have to prepend something to the function prototype to have it successfully found. For example the PETSc include files have these things __declspec(dllimport) The configure test fails because it does not provide the needed prototype. Likely you built PTScotch with static libraries so no problem.
The simplest fix would be to build static hypre libraries. I think it is a major project to get PETSc configure and macro system to work properly with external packages that are in Windows shared libraries since more use of __declspec would be needed.
Barry
The PETSc installation instructions should probably say something about external packages with Windows shared libraries.
On Jul 19, 2023, at 10:52 AM, Daniel Stone <[email protected]> wrote:
Hello,
I'm working on getting a petsc build running on windows. One necessary package to include is Hypre. I've been able to build Hypre seperately using cmake, and confirmed that the library works by setting up a VS project to run some of the example programs.
My attempted petsc build is being done through cygwin. I've been able to (with varying degrees of difficulty), build a fairly plain petsc, and one that downloads and builds ptscotch (after some modifications to both ptscotch and the config script). I am now attempting to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc). Note that the same compilers are being used for both Hypre and for petsc through cygwin - the new intel oneapi compilers (icx and ifx, after again varying amounts of pain to work around their awkwardness with the config script).
I'm seeing a problem when the config script does some tests on the included hypre lib. The source code looks like:
#include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */
#include "HYPRE.h"
char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main() { _check_HYPRE_IJMatrixCreate();; return 0; }
As I understand this is a fairly standard type of stub program used by the config script to check that it is able to link to certain symbols in given libraries. Tests like this have succeeded in my builds that include PTScotch.
I keep getting a linker error with the above test, including if I seperate it out and try to build it seperately:
unresolved external symbol "char __cdel HYPRE_IJMatrixCreate(void)" ....
Ok, it looks like a problem with either the library or linker commands. But here's the interesting thing - If I transplant this code into VS, with the same project setting that allows it to build the much more nontrivial Hypre example programs, I get the same error:
Error LNK2001 unresolved external symbol "char __cdecl HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1 C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1
So it seems like there might be something about this type of stub program that is not working with my Hypre library. I don't fully understand this program - it's able to call the function with no arguments, but it also needs to be linked against a library containing the function, apparently by wrapping it in a static void function? Not something I've seen before.
Does anyone have any insight into what might be going wrong - or really just any explaination of how the stub program works so I can figure out why it isn't in this case?
Many thanks,
Daniel
Satish, So it will always fail on Windows with Windows compilers (both with static and shared libraries)? Is this true for all PETSc external packages? If so, why does the installation documentation say that some external packages can work with Windows compilers? (Presumably PETSc cannot since the configure tests will fail). Barry
On Jul 19, 2023, at 11:40 AM, Satish Balay <[email protected]> wrote:
BTW: Some explanation of configure:
It attempts the following on linux:
Source: #include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */ char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } <<<<<<<
Note - it does not include 'HYPRE.h' here - but redefines the prototype as 'char HYPRE_IJMatrixCreate();
Compiling it manually:
[balay@pj01 petsc]$ cat conftest.c char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } [balay@pj01 petsc]$ gcc -c conftest.c [balay@pj01 petsc]$ nm -Ao conftest.o |grep HYPRE_IJMatrixCreate conftest.o:0000000000000000 t _check_HYPRE_IJMatrixCreate conftest.o: U HYPRE_IJMatrixCreate [balay@pj01 petsc]$ nm -Ao arch-linux-c-debug/lib/libHYPRE.so |grep HYPRE_IJMatrixCreate arch-linux-c-debug/lib/libHYPRE.so:000000000007f2c2 T HYPRE_IJMatrixCreate [balay@pj01 petsc]$ <<<<
Here the "U HYPRE_IJMatrixCreate" in conftest.o matches "T HYPRE_IJMatrixCreate" in libHYPRE.so - so the "link" test in configure succeeds!
[balay@pj01 petsc]$ gcc -o conftest conftest.o arch-linux-c-debug/lib/libHYPRE.so [balay@pj01 petsc]$ echo $? 0 <<<<<
On windows - [due to name mangling by cdecl/stdcall, (/MT vs /MD) etc..] - this might not match - resulting in link failures.
Satish
On Wed, 19 Jul 2023, Satish Balay via petsc-users wrote:
You could try skipping this test [and assume --with-hypre-include and --with-hypre-lib options are correct] - and see if this works.
diff --git a/config/BuildSystem/config/packages/hypre.py b/config/BuildSystem/config/packages/hypre.py index 5bc88322aa2..2d6c7932e17 100644 --- a/config/BuildSystem/config/packages/hypre.py +++ b/config/BuildSystem/config/packages/hypre.py @@ -11,7 +11,7 @@ class Configure(config.package.GNUPackage): self.requiresversion = 1 self.gitcommit = 'v'+self.version self.download = ['git://https://github.com/hypre-space/hypre','https://github.com/hypre-space/hypre/archive/'+self.gitcommit+'.tar.gz'] - self.functions = ['HYPRE_IJMatrixCreate'] + self.functions = [] self.includes = ['HYPRE.h'] self.liblist = [['libHYPRE.a']] self.buildLanguages = ['C','Cxx']
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
You don't indicate what type of libraries you built hypre with; static or shared. My guess is you ended up with shared
I think the answer to your difficulty is hidden in __cdecl (Satish will know much better than me). When you are looking for symbols in Windows shared libraries you have to prepend something to the function prototype to have it successfully found. For example the PETSc include files have these things __declspec(dllimport) The configure test fails because it does not provide the needed prototype. Likely you built PTScotch with static libraries so no problem.
The simplest fix would be to build static hypre libraries. I think it is a major project to get PETSc configure and macro system to work properly with external packages that are in Windows shared libraries since more use of __declspec would be needed.
Barry
The PETSc installation instructions should probably say something about external packages with Windows shared libraries.
On Jul 19, 2023, at 10:52 AM, Daniel Stone <[email protected]> wrote:
Hello,
I'm working on getting a petsc build running on windows. One necessary package to include is Hypre. I've been able to build Hypre seperately using cmake, and confirmed that the library works by setting up a VS project to run some of the example programs.
My attempted petsc build is being done through cygwin. I've been able to (with varying degrees of difficulty), build a fairly plain petsc, and one that downloads and builds ptscotch (after some modifications to both ptscotch and the config script). I am now attempting to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc). Note that the same compilers are being used for both Hypre and for petsc through cygwin - the new intel oneapi compilers (icx and ifx, after again varying amounts of pain to work around their awkwardness with the config script).
I'm seeing a problem when the config script does some tests on the included hypre lib. The source code looks like:
#include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */
#include "HYPRE.h"
char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main() { _check_HYPRE_IJMatrixCreate();; return 0; }
As I understand this is a fairly standard type of stub program used by the config script to check that it is able to link to certain symbols in given libraries. Tests like this have succeeded in my builds that include PTScotch.
I keep getting a linker error with the above test, including if I seperate it out and try to build it seperately:
unresolved external symbol "char __cdel HYPRE_IJMatrixCreate(void)" ....
Ok, it looks like a problem with either the library or linker commands. But here's the interesting thing - If I transplant this code into VS, with the same project setting that allows it to build the much more nontrivial Hypre example programs, I get the same error:
Error LNK2001 unresolved external symbol "char __cdecl HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1 C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1
So it seems like there might be something about this type of stub program that is not working with my Hypre library. I don't fully understand this program - it's able to call the function with no arguments, but it also needs to be linked against a library containing the function, apparently by wrapping it in a static void function? Not something I've seen before.
Does anyone have any insight into what might be going wrong - or really just any explaination of how the stub program works so I can figure out why it isn't in this case?
Many thanks,
Daniel
I think it should work with static libraries and 64bit compilers. That's how I think --download-f2cblaslapack [etc] work. Also it works with MS-MPI - even-though its a dll install, the library stub provides this symbol somehow.. balay@ps5 /cygdrive/c/Program Files (x86)/Microsoft SDKs/MPI/Lib/x64 $ nm -Ao msmpi.lib |grep " MPI_Init" msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init_thread msmpi.lib:msmpi.dll:0000000000000000 T MPI_Initialized However - if the library symbol is somehow mangled - this configure mode of checking library functions will fail. Checking PETSc dll build: balay@ps5 ~/petsc/arch-ci-mswin-uni/lib $ nm -Ao libpetsc.lib |grep MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 I __imp_MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 T MatCreateSeqAIJWithArrays It also has the unmangled symbol - so I guess this mode can work generally with dlls. Satish On Wed, 19 Jul 2023, Barry Smith wrote:
Satish,
So it will always fail on Windows with Windows compilers (both with static and shared libraries)? Is this true for all PETSc external packages? If so, why does the installation documentation say that some external packages can work with Windows compilers? (Presumably PETSc cannot since the configure tests will fail).
Barry
On Jul 19, 2023, at 11:40 AM, Satish Balay <[email protected]> wrote:
BTW: Some explanation of configure:
It attempts the following on linux:
Source: #include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */ char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } <<<<<<<
Note - it does not include 'HYPRE.h' here - but redefines the prototype as 'char HYPRE_IJMatrixCreate();
Compiling it manually:
[balay@pj01 petsc]$ cat conftest.c char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } [balay@pj01 petsc]$ gcc -c conftest.c [balay@pj01 petsc]$ nm -Ao conftest.o |grep HYPRE_IJMatrixCreate conftest.o:0000000000000000 t _check_HYPRE_IJMatrixCreate conftest.o: U HYPRE_IJMatrixCreate [balay@pj01 petsc]$ nm -Ao arch-linux-c-debug/lib/libHYPRE.so |grep HYPRE_IJMatrixCreate arch-linux-c-debug/lib/libHYPRE.so:000000000007f2c2 T HYPRE_IJMatrixCreate [balay@pj01 petsc]$ <<<<
Here the "U HYPRE_IJMatrixCreate" in conftest.o matches "T HYPRE_IJMatrixCreate" in libHYPRE.so - so the "link" test in configure succeeds!
[balay@pj01 petsc]$ gcc -o conftest conftest.o arch-linux-c-debug/lib/libHYPRE.so [balay@pj01 petsc]$ echo $? 0 <<<<<
On windows - [due to name mangling by cdecl/stdcall, (/MT vs /MD) etc..] - this might not match - resulting in link failures.
Satish
On Wed, 19 Jul 2023, Satish Balay via petsc-users wrote:
You could try skipping this test [and assume --with-hypre-include and --with-hypre-lib options are correct] - and see if this works.
diff --git a/config/BuildSystem/config/packages/hypre.py b/config/BuildSystem/config/packages/hypre.py index 5bc88322aa2..2d6c7932e17 100644 --- a/config/BuildSystem/config/packages/hypre.py +++ b/config/BuildSystem/config/packages/hypre.py @@ -11,7 +11,7 @@ class Configure(config.package.GNUPackage): self.requiresversion = 1 self.gitcommit = 'v'+self.version self.download = ['git://https://github.com/hypre-space/hypre','https://github.com/hypre-space/hypre/archive/'+self.gitcommit+'.tar.gz'] - self.functions = ['HYPRE_IJMatrixCreate'] + self.functions = [] self.includes = ['HYPRE.h'] self.liblist = [['libHYPRE.a']] self.buildLanguages = ['C','Cxx']
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
You don't indicate what type of libraries you built hypre with; static or shared. My guess is you ended up with shared
I think the answer to your difficulty is hidden in __cdecl (Satish will know much better than me). When you are looking for symbols in Windows shared libraries you have to prepend something to the function prototype to have it successfully found. For example the PETSc include files have these things __declspec(dllimport) The configure test fails because it does not provide the needed prototype. Likely you built PTScotch with static libraries so no problem.
The simplest fix would be to build static hypre libraries. I think it is a major project to get PETSc configure and macro system to work properly with external packages that are in Windows shared libraries since more use of __declspec would be needed.
Barry
The PETSc installation instructions should probably say something about external packages with Windows shared libraries.
On Jul 19, 2023, at 10:52 AM, Daniel Stone <[email protected]> wrote:
Hello,
I'm working on getting a petsc build running on windows. One necessary package to include is Hypre. I've been able to build Hypre seperately using cmake, and confirmed that the library works by setting up a VS project to run some of the example programs.
My attempted petsc build is being done through cygwin. I've been able to (with varying degrees of difficulty), build a fairly plain petsc, and one that downloads and builds ptscotch (after some modifications to both ptscotch and the config script). I am now attempting to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc). Note that the same compilers are being used for both Hypre and for petsc through cygwin - the new intel oneapi compilers (icx and ifx, after again varying amounts of pain to work around their awkwardness with the config script).
I'm seeing a problem when the config script does some tests on the included hypre lib. The source code looks like:
#include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */
#include "HYPRE.h"
char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main() { _check_HYPRE_IJMatrixCreate();; return 0; }
As I understand this is a fairly standard type of stub program used by the config script to check that it is able to link to certain symbols in given libraries. Tests like this have succeeded in my builds that include PTScotch.
I keep getting a linker error with the above test, including if I seperate it out and try to build it seperately:
unresolved external symbol "char __cdel HYPRE_IJMatrixCreate(void)" ....
Ok, it looks like a problem with either the library or linker commands. But here's the interesting thing - If I transplant this code into VS, with the same project setting that allows it to build the much more nontrivial Hypre example programs, I get the same error:
Error LNK2001 unresolved external symbol "char __cdecl HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1 C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1
So it seems like there might be something about this type of stub program that is not working with my Hypre library. I don't fully understand this program - it's able to call the function with no arguments, but it also needs to be linked against a library containing the function, apparently by wrapping it in a static void function? Not something I've seen before.
Does anyone have any insight into what might be going wrong - or really just any explaination of how the stub program works so I can figure out why it isn't in this case?
Many thanks,
Daniel
Hi All, Many thanks for the detailed explainations and ideas! I tried skipping the test. When it came time to do the build itself (make $PETSC_DIR... all) I get some failures, unsurprisingly: -------------------------------- FC arch-mswin-c-opt/obj/dm/f90-mod/petscdmplexmod.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-custom/zhypref.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-auto/hypref.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(444,29): error: assigning to 'hypre_Error' from incompatible type 'int' hypre__global_error = 0; ^ ~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7): note: expanded from macro 'PetscStackCallExternalVoid' __VA_ARGS__; \ ^~~~~~~~~~~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(634,29): error: assigning to 'hypre_Error' from incompatible type 'int' hypre__global_error = 0; ^ ~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7): note: expanded from macro 'PetscStackCallExternalVoid' __VA_ARGS__; \ ^~~~~~~~~~~ 2 errors generated. make[3]: *** [gmakefile:195: arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o] Error 1 make[3]: *** Waiting for unfinished jobs.... FC arch-mswin-c-opt/obj/ksp/f90-mod/petsckspdefmod.o CC arch-mswin-c-opt/obj/dm/impls/da/hypre/mhyp.o CC arch-mswin-c-opt/obj/mat/impls/hypre/mhypre.o make[3]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' make[2]: *** [/home/DanielOGS/petsc_ogs_3.19.1/lib/petsc/conf/rules.doc:28: libs] Error 2 make[2]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' **************************ERROR************************************* Error during compile, check arch-mswin-c-opt/lib/petsc/conf/make.log Send it and arch-mswin-c-opt/lib/petsc/conf/configure.log to [email protected] ******************************************************************** Finishing make run at Wed, 19 Jul 2023 17:07:00 +0100 ---------------------------------------- But wait - isn't this the compile stage, not the linking stage? This seems to imply that I've made a hash of providing include file such that a definition of "hypre_Error" cannot be seen - unless I'm misinterpreting. Interesting note about Hypre and include files - if built using configure and make, all the include files are conviniently copied into hypre/src/hypre/include. This is not done for a cmake build - I had to do the copying myself. Maybe I missed one. On shared vs. static - if there a clear way of telling which I've ended up with? I've checked the cmakelists for hypre and this seems to imply that not-shared is the default, which I didn't change: # Configuration options option(HYPRE_ENABLE_SHARED "Build a shared library" OFF) option(HYPRE_ENABLE_BIGINT "Use long long int for HYPRE_Int" OFF) option(HYPRE_ENABLE_MIXEDINT "Use long long int for HYPRE_BigInt, int for HYPRE_INT" OFF) [....] checking again, I've noticed that the way that the stub-test fails is different depending on whether it's called from the config script or used in isolation - more details on that soon. Thanks again, Daniel On Wed, Jul 19, 2023 at 4:58 PM Satish Balay via petsc-users < [email protected]> wrote:
I think it should work with static libraries and 64bit compilers.
That's how I think --download-f2cblaslapack [etc] work.
Also it works with MS-MPI - even-though its a dll install, the library stub provides this symbol somehow..
balay@ps5 /cygdrive/c/Program Files (x86)/Microsoft SDKs/MPI/Lib/x64 $ nm -Ao msmpi.lib |grep " MPI_Init" msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init_thread msmpi.lib:msmpi.dll:0000000000000000 T MPI_Initialized
However - if the library symbol is somehow mangled - this configure mode of checking library functions will fail.
Checking PETSc dll build:
balay@ps5 ~/petsc/arch-ci-mswin-uni/lib $ nm -Ao libpetsc.lib |grep MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 I __imp_MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 T MatCreateSeqAIJWithArrays
It also has the unmangled symbol - so I guess this mode can work generally with dlls.
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
Satish,
So it will always fail on Windows with Windows compilers (both with
static and shared libraries)? Is this true for all PETSc external packages? If so, why does the installation documentation say that some external packages can work with Windows compilers? (Presumably PETSc cannot since the configure tests will fail).
Barry
On Jul 19, 2023, at 11:40 AM, Satish Balay <[email protected]> wrote:
BTW: Some explanation of configure:
It attempts the following on linux:
Source: #include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */ char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } <<<<<<<
Note - it does not include 'HYPRE.h' here - but redefines the
prototype as 'char HYPRE_IJMatrixCreate();
Compiling it manually:
[balay@pj01 petsc]$ cat conftest.c char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } [balay@pj01 petsc]$ gcc -c conftest.c [balay@pj01 petsc]$ nm -Ao conftest.o |grep HYPRE_IJMatrixCreate conftest.o:0000000000000000 t _check_HYPRE_IJMatrixCreate conftest.o: U HYPRE_IJMatrixCreate [balay@pj01 petsc]$ nm -Ao arch-linux-c-debug/lib/libHYPRE.so |grep
HYPRE_IJMatrixCreate
arch-linux-c-debug/lib/libHYPRE.so:000000000007f2c2 T HYPRE_IJMatrixCreate [balay@pj01 petsc]$ <<<<
Here the "U HYPRE_IJMatrixCreate" in conftest.o matches "T HYPRE_IJMatrixCreate" in libHYPRE.so - so the "link" test in configure succeeds!
[balay@pj01 petsc]$ gcc -o conftest conftest.o arch-linux-c-debug/lib/libHYPRE.so [balay@pj01 petsc]$ echo $? 0 <<<<<
On windows - [due to name mangling by cdecl/stdcall, (/MT vs /MD) etc..] - this might not match - resulting in link failures.
Satish
On Wed, 19 Jul 2023, Satish Balay via petsc-users wrote:
You could try skipping this test [and assume --with-hypre-include and --with-hypre-lib options are correct] - and see if this works.
diff --git a/config/BuildSystem/config/packages/hypre.py b/config/BuildSystem/config/packages/hypre.py index 5bc88322aa2..2d6c7932e17 100644 --- a/config/BuildSystem/config/packages/hypre.py +++ b/config/BuildSystem/config/packages/hypre.py @@ -11,7 +11,7 @@ class Configure(config.package.GNUPackage): self.requiresversion = 1 self.gitcommit = 'v'+self.version self.download = ['git:// https://github.com/hypre-space/hypre',' https://github.com/hypre-space/hypre/archive/'+self.gitcommit+'.tar.gz'] - self.functions = ['HYPRE_IJMatrixCreate'] + self.functions = [] self.includes = ['HYPRE.h'] self.liblist = [['libHYPRE.a']] self.buildLanguages = ['C','Cxx']
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
You don't indicate what type of libraries you built hypre with;
static or shared. My guess is you ended up with shared
I think the answer to your difficulty is hidden in __cdecl (Satish
will know much better than me). When you are looking for symbols in Windows shared libraries you have to prepend something to the function prototype to have it successfully found. For example the PETSc include files have these things __declspec(dllimport) The configure test fails because it does not provide the needed prototype. Likely you built PTScotch with static libraries so no problem.
The simplest fix would be to build static hypre libraries. I think
it is a major project to get PETSc configure and macro system to work properly with external packages that are in Windows shared libraries since more use of __declspec would be needed.
Barry
The PETSc installation instructions should probably say something
about external packages with Windows shared libraries.
On Jul 19, 2023, at 10:52 AM, Daniel Stone <
[email protected]> wrote:
Hello,
I'm working on getting a petsc build running on windows. One
necessary package to include is Hypre. I've been able to build Hypre seperately using cmake, and confirmed that the library works
by setting up a VS project to run some of the example programs.
My attempted petsc build is being done through cygwin. I've been able to (with varying degrees of difficulty), build a fairly plain petsc, and one that downloads and builds ptscotch (after some modifications to both ptscotch and the config script). I am now attempting to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc). Note that the same compilers are being used for both Hypre and for petsc through cygwin - the new intel oneapi compilers (icx and ifx, after again varying amounts of pain to work around their awkwardness with the config script).
I'm seeing a problem when the config script does some tests on the included hypre lib. The source code looks like:
#include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */
#include "HYPRE.h"
char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main() { _check_HYPRE_IJMatrixCreate();; return 0; }
As I understand this is a fairly standard type of stub program used by the config script to check that it is able to link to certain symbols in given libraries. Tests like this have succeeded in my builds that include PTScotch.
I keep getting a linker error with the above test, including if I seperate it out and try to build it seperately:
unresolved external symbol "char __cdel HYPRE_IJMatrixCreate(void)" ....
Ok, it looks like a problem with either the library or linker commands. But here's the interesting thing - If I transplant this code into VS, with the same project setting that allows it to build the much more nontrivial Hypre example programs, I get the same error:
Error LNK2001 unresolved external symbol "char __cdecl HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1 C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1
So it seems like there might be something about this type of stub program that is not working with my Hypre library. I don't fully understand this program - it's able to call the function with no arguments, but it also needs to be linked against a library containing the function, apparently by wrapping it in a static void function? Not something I've seen before.
Does anyone have any insight into what might be going wrong - or really just any explaination of how the stub program works so I can figure out why it isn't in this case?
Many thanks,
Daniel
You cannot use this version of PETSc, 3.19, with the version of hypre you installed. In hypre they recently changed hypre_Error from an integer to a struct which completely breaks compatibility with previous versions of hypre (and hence previous versions of PETSc). You must use the main git branch of PETSc with the version of hypre you installed. Barry
On Jul 20, 2023, at 5:10 AM, Daniel Stone <[email protected]> wrote:
Hi All,
Many thanks for the detailed explainations and ideas!
I tried skipping the test. When it came time to do the build itself (make $PETSC_DIR... all) I get some failures, unsurprisingly:
--------------------------------
FC arch-mswin-c-opt/obj/dm/f90-mod/petscdmplexmod.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-custom/zhypref.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-auto/hypref.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(444,29): error: assigning to 'hypre_Error' from incompatible type 'int' hypre__global_error = 0; ^ ~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7): note: expanded from macro 'PetscStackCallExternalVoid' __VA_ARGS__; \ ^~~~~~~~~~~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(634,29): error: assigning to 'hypre_Error' from incompatible type 'int' hypre__global_error = 0; ^ ~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7): note: expanded from macro 'PetscStackCallExternalVoid' __VA_ARGS__; \ ^~~~~~~~~~~ 2 errors generated. make[3]: *** [gmakefile:195: arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o] Error 1 make[3]: *** Waiting for unfinished jobs.... FC arch-mswin-c-opt/obj/ksp/f90-mod/petsckspdefmod.o CC arch-mswin-c-opt/obj/dm/impls/da/hypre/mhyp.o CC arch-mswin-c-opt/obj/mat/impls/hypre/mhypre.o make[3]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' make[2]: *** [/home/DanielOGS/petsc_ogs_3.19.1/lib/petsc/conf/rules.doc:28: libs] Error 2 make[2]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' **************************ERROR************************************* Error during compile, check arch-mswin-c-opt/lib/petsc/conf/make.log Send it and arch-mswin-c-opt/lib/petsc/conf/configure.log to [email protected] <mailto:[email protected]> ******************************************************************** Finishing make run at Wed, 19 Jul 2023 17:07:00 +0100
----------------------------------------
But wait - isn't this the compile stage, not the linking stage? This seems to imply that I've made a hash of providing include file such that a definition of "hypre_Error" cannot be seen - unless I'm misinterpreting. Interesting note about Hypre and include files - if built using configure and make, all the include files are conviniently copied into hypre/src/hypre/include. This is not done for a cmake build - I had to do the copying myself. Maybe I missed one.
On shared vs. static - if there a clear way of telling which I've ended up with? I've checked the cmakelists for hypre and this seems to imply that not-shared is the default, which I didn't change:
# Configuration options option(HYPRE_ENABLE_SHARED "Build a shared library" OFF) option(HYPRE_ENABLE_BIGINT "Use long long int for HYPRE_Int" OFF) option(HYPRE_ENABLE_MIXEDINT "Use long long int for HYPRE_BigInt, int for HYPRE_INT" OFF) [....]
checking again, I've noticed that the way that the stub-test fails is different depending on whether it's called from the config script or used in isolation - more details on that soon.
Thanks again,
Daniel
On Wed, Jul 19, 2023 at 4:58 PM Satish Balay via petsc-users <[email protected] <mailto:[email protected]>> wrote:
I think it should work with static libraries and 64bit compilers.
That's how I think --download-f2cblaslapack [etc] work.
Also it works with MS-MPI - even-though its a dll install, the library stub provides this symbol somehow..
balay@ps5 /cygdrive/c/Program Files (x86)/Microsoft SDKs/MPI/Lib/x64 $ nm -Ao msmpi.lib |grep " MPI_Init" msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init_thread msmpi.lib:msmpi.dll:0000000000000000 T MPI_Initialized
However - if the library symbol is somehow mangled - this configure mode of checking library functions will fail.
Checking PETSc dll build:
balay@ps5 ~/petsc/arch-ci-mswin-uni/lib $ nm -Ao libpetsc.lib |grep MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 I __imp_MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 T MatCreateSeqAIJWithArrays
It also has the unmangled symbol - so I guess this mode can work generally with dlls.
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
Satish,
So it will always fail on Windows with Windows compilers (both with static and shared libraries)? Is this true for all PETSc external packages? If so, why does the installation documentation say that some external packages can work with Windows compilers? (Presumably PETSc cannot since the configure tests will fail).
Barry
On Jul 19, 2023, at 11:40 AM, Satish Balay <[email protected] <mailto:[email protected]>> wrote:
BTW: Some explanation of configure:
It attempts the following on linux:
Source: #include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */ char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } <<<<<<<
Note - it does not include 'HYPRE.h' here - but redefines the prototype as 'char HYPRE_IJMatrixCreate();
Compiling it manually:
[balay@pj01 petsc]$ cat conftest.c char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } [balay@pj01 petsc]$ gcc -c conftest.c [balay@pj01 petsc]$ nm -Ao conftest.o |grep HYPRE_IJMatrixCreate conftest.o:0000000000000000 t _check_HYPRE_IJMatrixCreate conftest.o: U HYPRE_IJMatrixCreate [balay@pj01 petsc]$ nm -Ao arch-linux-c-debug/lib/libHYPRE.so |grep HYPRE_IJMatrixCreate arch-linux-c-debug/lib/libHYPRE.so:000000000007f2c2 T HYPRE_IJMatrixCreate [balay@pj01 petsc]$ <<<<
Here the "U HYPRE_IJMatrixCreate" in conftest.o matches "T HYPRE_IJMatrixCreate" in libHYPRE.so - so the "link" test in configure succeeds!
[balay@pj01 petsc]$ gcc -o conftest conftest.o arch-linux-c-debug/lib/libHYPRE.so [balay@pj01 petsc]$ echo $? 0 <<<<<
On windows - [due to name mangling by cdecl/stdcall, (/MT vs /MD) etc..] - this might not match - resulting in link failures.
Satish
On Wed, 19 Jul 2023, Satish Balay via petsc-users wrote:
You could try skipping this test [and assume --with-hypre-include and --with-hypre-lib options are correct] - and see if this works.
diff --git a/config/BuildSystem/config/packages/hypre.py b/config/BuildSystem/config/packages/hypre.py index 5bc88322aa2..2d6c7932e17 100644 --- a/config/BuildSystem/config/packages/hypre.py +++ b/config/BuildSystem/config/packages/hypre.py @@ -11,7 +11,7 @@ class Configure(config.package.GNUPackage): self.requiresversion = 1 self.gitcommit = 'v'+self.version self.download = ['git://https://github.com/hypre-space/hypre','https://github.com/hypre-space/hypre/archive/'+self.gitcommit+'.tar.gz'] - self.functions = ['HYPRE_IJMatrixCreate'] + self.functions = [] self.includes = ['HYPRE.h'] self.liblist = [['libHYPRE.a']] self.buildLanguages = ['C','Cxx']
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
You don't indicate what type of libraries you built hypre with; static or shared. My guess is you ended up with shared
I think the answer to your difficulty is hidden in __cdecl (Satish will know much better than me). When you are looking for symbols in Windows shared libraries you have to prepend something to the function prototype to have it successfully found. For example the PETSc include files have these things __declspec(dllimport) The configure test fails because it does not provide the needed prototype. Likely you built PTScotch with static libraries so no problem.
The simplest fix would be to build static hypre libraries. I think it is a major project to get PETSc configure and macro system to work properly with external packages that are in Windows shared libraries since more use of __declspec would be needed.
Barry
The PETSc installation instructions should probably say something about external packages with Windows shared libraries.
On Jul 19, 2023, at 10:52 AM, Daniel Stone <[email protected] <mailto:[email protected]>> wrote:
Hello,
I'm working on getting a petsc build running on windows. One necessary package to include is Hypre. I've been able to build Hypre seperately using cmake, and confirmed that the library works by setting up a VS project to run some of the example programs.
My attempted petsc build is being done through cygwin. I've been able to (with varying degrees of difficulty), build a fairly plain petsc, and one that downloads and builds ptscotch (after some modifications to both ptscotch and the config script). I am now attempting to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc). Note that the same compilers are being used for both Hypre and for petsc through cygwin - the new intel oneapi compilers (icx and ifx, after again varying amounts of pain to work around their awkwardness with the config script).
I'm seeing a problem when the config script does some tests on the included hypre lib. The source code looks like:
#include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */
#include "HYPRE.h"
char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main() { _check_HYPRE_IJMatrixCreate();; return 0; }
As I understand this is a fairly standard type of stub program used by the config script to check that it is able to link to certain symbols in given libraries. Tests like this have succeeded in my builds that include PTScotch.
I keep getting a linker error with the above test, including if I seperate it out and try to build it seperately:
unresolved external symbol "char __cdel HYPRE_IJMatrixCreate(void)" ....
Ok, it looks like a problem with either the library or linker commands. But here's the interesting thing - If I transplant this code into VS, with the same project setting that allows it to build the much more nontrivial Hypre example programs, I get the same error:
Error LNK2001 unresolved external symbol "char __cdecl HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1 C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1
So it seems like there might be something about this type of stub program that is not working with my Hypre library. I don't fully understand this program - it's able to call the function with no arguments, but it also needs to be linked against a library containing the function, apparently by wrapping it in a static void function? Not something I've seen before.
Does anyone have any insight into what might be going wrong - or really just any explaination of how the stub program works so I can figure out why it isn't in this case?
Many thanks,
Daniel
Can check config/BuildSystem/config/packages/hypre.py petsc-3.19 (or release branch) is compatible with hypre 2.28.0, petsc 'main' branch with 2.29.0 Satish On Thu, 20 Jul 2023, Barry Smith via petsc-users wrote:
You cannot use this version of PETSc, 3.19, with the version of hypre you installed. In hypre they recently changed hypre_Error from an integer to a struct which completely breaks compatibility with previous versions of hypre (and hence previous versions of PETSc). You must use the main git branch of PETSc with the version of hypre you installed.
Barry
On Jul 20, 2023, at 5:10 AM, Daniel Stone <[email protected]> wrote:
Hi All,
Many thanks for the detailed explainations and ideas!
I tried skipping the test. When it came time to do the build itself (make $PETSC_DIR... all) I get some failures, unsurprisingly:
--------------------------------
FC arch-mswin-c-opt/obj/dm/f90-mod/petscdmplexmod.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-custom/zhypref.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-auto/hypref.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(444,29): error: assigning to 'hypre_Error' from incompatible type 'int' hypre__global_error = 0; ^ ~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7): note: expanded from macro 'PetscStackCallExternalVoid' __VA_ARGS__; \ ^~~~~~~~~~~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(634,29): error: assigning to 'hypre_Error' from incompatible type 'int' hypre__global_error = 0; ^ ~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7): note: expanded from macro 'PetscStackCallExternalVoid' __VA_ARGS__; \ ^~~~~~~~~~~ 2 errors generated. make[3]: *** [gmakefile:195: arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o] Error 1 make[3]: *** Waiting for unfinished jobs.... FC arch-mswin-c-opt/obj/ksp/f90-mod/petsckspdefmod.o CC arch-mswin-c-opt/obj/dm/impls/da/hypre/mhyp.o CC arch-mswin-c-opt/obj/mat/impls/hypre/mhypre.o make[3]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' make[2]: *** [/home/DanielOGS/petsc_ogs_3.19.1/lib/petsc/conf/rules.doc:28: libs] Error 2 make[2]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' **************************ERROR************************************* Error during compile, check arch-mswin-c-opt/lib/petsc/conf/make.log Send it and arch-mswin-c-opt/lib/petsc/conf/configure.log to [email protected] <mailto:[email protected]> ******************************************************************** Finishing make run at Wed, 19 Jul 2023 17:07:00 +0100
----------------------------------------
But wait - isn't this the compile stage, not the linking stage? This seems to imply that I've made a hash of providing include file such that a definition of "hypre_Error" cannot be seen - unless I'm misinterpreting. Interesting note about Hypre and include files - if built using configure and make, all the include files are conviniently copied into hypre/src/hypre/include. This is not done for a cmake build - I had to do the copying myself. Maybe I missed one.
On shared vs. static - if there a clear way of telling which I've ended up with? I've checked the cmakelists for hypre and this seems to imply that not-shared is the default, which I didn't change:
# Configuration options option(HYPRE_ENABLE_SHARED "Build a shared library" OFF) option(HYPRE_ENABLE_BIGINT "Use long long int for HYPRE_Int" OFF) option(HYPRE_ENABLE_MIXEDINT "Use long long int for HYPRE_BigInt, int for HYPRE_INT" OFF) [....]
checking again, I've noticed that the way that the stub-test fails is different depending on whether it's called from the config script or used in isolation - more details on that soon.
Thanks again,
Daniel
On Wed, Jul 19, 2023 at 4:58 PM Satish Balay via petsc-users <[email protected] <mailto:[email protected]>> wrote:
I think it should work with static libraries and 64bit compilers.
That's how I think --download-f2cblaslapack [etc] work.
Also it works with MS-MPI - even-though its a dll install, the library stub provides this symbol somehow..
balay@ps5 /cygdrive/c/Program Files (x86)/Microsoft SDKs/MPI/Lib/x64 $ nm -Ao msmpi.lib |grep " MPI_Init" msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init_thread msmpi.lib:msmpi.dll:0000000000000000 T MPI_Initialized
However - if the library symbol is somehow mangled - this configure mode of checking library functions will fail.
Checking PETSc dll build:
balay@ps5 ~/petsc/arch-ci-mswin-uni/lib $ nm -Ao libpetsc.lib |grep MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 I __imp_MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 T MatCreateSeqAIJWithArrays
It also has the unmangled symbol - so I guess this mode can work generally with dlls.
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
Satish,
So it will always fail on Windows with Windows compilers (both with static and shared libraries)? Is this true for all PETSc external packages? If so, why does the installation documentation say that some external packages can work with Windows compilers? (Presumably PETSc cannot since the configure tests will fail).
Barry
On Jul 19, 2023, at 11:40 AM, Satish Balay <[email protected] <mailto:[email protected]>> wrote:
BTW: Some explanation of configure:
It attempts the following on linux:
Source: #include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */ char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } <<<<<<<
Note - it does not include 'HYPRE.h' here - but redefines the prototype as 'char HYPRE_IJMatrixCreate();
Compiling it manually:
[balay@pj01 petsc]$ cat conftest.c char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } [balay@pj01 petsc]$ gcc -c conftest.c [balay@pj01 petsc]$ nm -Ao conftest.o |grep HYPRE_IJMatrixCreate conftest.o:0000000000000000 t _check_HYPRE_IJMatrixCreate conftest.o: U HYPRE_IJMatrixCreate [balay@pj01 petsc]$ nm -Ao arch-linux-c-debug/lib/libHYPRE.so |grep HYPRE_IJMatrixCreate arch-linux-c-debug/lib/libHYPRE.so:000000000007f2c2 T HYPRE_IJMatrixCreate [balay@pj01 petsc]$ <<<<
Here the "U HYPRE_IJMatrixCreate" in conftest.o matches "T HYPRE_IJMatrixCreate" in libHYPRE.so - so the "link" test in configure succeeds!
[balay@pj01 petsc]$ gcc -o conftest conftest.o arch-linux-c-debug/lib/libHYPRE.so [balay@pj01 petsc]$ echo $? 0 <<<<<
On windows - [due to name mangling by cdecl/stdcall, (/MT vs /MD) etc..] - this might not match - resulting in link failures.
Satish
On Wed, 19 Jul 2023, Satish Balay via petsc-users wrote:
You could try skipping this test [and assume --with-hypre-include and --with-hypre-lib options are correct] - and see if this works.
diff --git a/config/BuildSystem/config/packages/hypre.py b/config/BuildSystem/config/packages/hypre.py index 5bc88322aa2..2d6c7932e17 100644 --- a/config/BuildSystem/config/packages/hypre.py +++ b/config/BuildSystem/config/packages/hypre.py @@ -11,7 +11,7 @@ class Configure(config.package.GNUPackage): self.requiresversion = 1 self.gitcommit = 'v'+self.version self.download = ['git://https://github.com/hypre-space/hypre','https://github.com/hypre-space/hypre/archive/'+self.gitcommit+'.tar.gz'] - self.functions = ['HYPRE_IJMatrixCreate'] + self.functions = [] self.includes = ['HYPRE.h'] self.liblist = [['libHYPRE.a']] self.buildLanguages = ['C','Cxx']
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
You don't indicate what type of libraries you built hypre with; static or shared. My guess is you ended up with shared
I think the answer to your difficulty is hidden in __cdecl (Satish will know much better than me). When you are looking for symbols in Windows shared libraries you have to prepend something to the function prototype to have it successfully found. For example the PETSc include files have these things __declspec(dllimport) The configure test fails because it does not provide the needed prototype. Likely you built PTScotch with static libraries so no problem.
The simplest fix would be to build static hypre libraries. I think it is a major project to get PETSc configure and macro system to work properly with external packages that are in Windows shared libraries since more use of __declspec would be needed.
Barry
The PETSc installation instructions should probably say something about external packages with Windows shared libraries.
On Jul 19, 2023, at 10:52 AM, Daniel Stone <[email protected] <mailto:[email protected]>> wrote:
Hello,
I'm working on getting a petsc build running on windows. One necessary package to include is Hypre. I've been able to build Hypre seperately using cmake, and confirmed that the library works by setting up a VS project to run some of the example programs.
My attempted petsc build is being done through cygwin. I've been able to (with varying degrees of difficulty), build a fairly plain petsc, and one that downloads and builds ptscotch (after some modifications to both ptscotch and the config script). I am now attempting to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc). Note that the same compilers are being used for both Hypre and for petsc through cygwin - the new intel oneapi compilers (icx and ifx, after again varying amounts of pain to work around their awkwardness with the config script).
I'm seeing a problem when the config script does some tests on the included hypre lib. The source code looks like:
#include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */
#include "HYPRE.h"
char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main() { _check_HYPRE_IJMatrixCreate();; return 0; }
As I understand this is a fairly standard type of stub program used by the config script to check that it is able to link to certain symbols in given libraries. Tests like this have succeeded in my builds that include PTScotch.
I keep getting a linker error with the above test, including if I seperate it out and try to build it seperately:
unresolved external symbol "char __cdel HYPRE_IJMatrixCreate(void)" ....
Ok, it looks like a problem with either the library or linker commands. But here's the interesting thing - If I transplant this code into VS, with the same project setting that allows it to build the much more nontrivial Hypre example programs, I get the same error:
Error LNK2001 unresolved external symbol "char __cdecl HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1 C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1
So it seems like there might be something about this type of stub program that is not working with my Hypre library. I don't fully understand this program - it's able to call the function with no arguments, but it also needs to be linked against a library containing the function, apparently by wrapping it in a static void function? Not something I've seen before.
Does anyone have any insight into what might be going wrong - or really just any explaination of how the stub program works so I can figure out why it isn't in this case?
Many thanks,
Daniel
Satish, I think hypre.py in main needs minversion = 2.29 instead of 2.14
On Jul 20, 2023, at 11:05 AM, Satish Balay <[email protected]> wrote:
Can check config/BuildSystem/config/packages/hypre.py
petsc-3.19 (or release branch) is compatible with hypre 2.28.0, petsc 'main' branch with 2.29.0
Satish
On Thu, 20 Jul 2023, Barry Smith via petsc-users wrote:
You cannot use this version of PETSc, 3.19, with the version of hypre you installed. In hypre they recently changed hypre_Error from an integer to a struct which completely breaks compatibility with previous versions of hypre (and hence previous versions of PETSc). You must use the main git branch of PETSc with the version of hypre you installed.
Barry
On Jul 20, 2023, at 5:10 AM, Daniel Stone <[email protected]> wrote:
Hi All,
Many thanks for the detailed explainations and ideas!
I tried skipping the test. When it came time to do the build itself (make $PETSC_DIR... all) I get some failures, unsurprisingly:
--------------------------------
FC arch-mswin-c-opt/obj/dm/f90-mod/petscdmplexmod.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-custom/zhypref.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-auto/hypref.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(444,29): error: assigning to 'hypre_Error' from incompatible type 'int' hypre__global_error = 0; ^ ~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7): note: expanded from macro 'PetscStackCallExternalVoid' __VA_ARGS__; \ ^~~~~~~~~~~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(634,29): error: assigning to 'hypre_Error' from incompatible type 'int' hypre__global_error = 0; ^ ~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7): note: expanded from macro 'PetscStackCallExternalVoid' __VA_ARGS__; \ ^~~~~~~~~~~ 2 errors generated. make[3]: *** [gmakefile:195: arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o] Error 1 make[3]: *** Waiting for unfinished jobs.... FC arch-mswin-c-opt/obj/ksp/f90-mod/petsckspdefmod.o CC arch-mswin-c-opt/obj/dm/impls/da/hypre/mhyp.o CC arch-mswin-c-opt/obj/mat/impls/hypre/mhypre.o make[3]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' make[2]: *** [/home/DanielOGS/petsc_ogs_3.19.1/lib/petsc/conf/rules.doc:28: libs] Error 2 make[2]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' **************************ERROR************************************* Error during compile, check arch-mswin-c-opt/lib/petsc/conf/make.log Send it and arch-mswin-c-opt/lib/petsc/conf/configure.log to [email protected] <mailto:[email protected]> ******************************************************************** Finishing make run at Wed, 19 Jul 2023 17:07:00 +0100
----------------------------------------
But wait - isn't this the compile stage, not the linking stage? This seems to imply that I've made a hash of providing include file such that a definition of "hypre_Error" cannot be seen - unless I'm misinterpreting. Interesting note about Hypre and include files - if built using configure and make, all the include files are conviniently copied into hypre/src/hypre/include. This is not done for a cmake build - I had to do the copying myself. Maybe I missed one.
On shared vs. static - if there a clear way of telling which I've ended up with? I've checked the cmakelists for hypre and this seems to imply that not-shared is the default, which I didn't change:
# Configuration options option(HYPRE_ENABLE_SHARED "Build a shared library" OFF) option(HYPRE_ENABLE_BIGINT "Use long long int for HYPRE_Int" OFF) option(HYPRE_ENABLE_MIXEDINT "Use long long int for HYPRE_BigInt, int for HYPRE_INT" OFF) [....]
checking again, I've noticed that the way that the stub-test fails is different depending on whether it's called from the config script or used in isolation - more details on that soon.
Thanks again,
Daniel
On Wed, Jul 19, 2023 at 4:58 PM Satish Balay via petsc-users <[email protected] <mailto:[email protected]>> wrote:
I think it should work with static libraries and 64bit compilers.
That's how I think --download-f2cblaslapack [etc] work.
Also it works with MS-MPI - even-though its a dll install, the library stub provides this symbol somehow..
balay@ps5 /cygdrive/c/Program Files (x86)/Microsoft SDKs/MPI/Lib/x64 $ nm -Ao msmpi.lib |grep " MPI_Init" msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init_thread msmpi.lib:msmpi.dll:0000000000000000 T MPI_Initialized
However - if the library symbol is somehow mangled - this configure mode of checking library functions will fail.
Checking PETSc dll build:
balay@ps5 ~/petsc/arch-ci-mswin-uni/lib $ nm -Ao libpetsc.lib |grep MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 I __imp_MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 T MatCreateSeqAIJWithArrays
It also has the unmangled symbol - so I guess this mode can work generally with dlls.
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
Satish,
So it will always fail on Windows with Windows compilers (both with static and shared libraries)? Is this true for all PETSc external packages? If so, why does the installation documentation say that some external packages can work with Windows compilers? (Presumably PETSc cannot since the configure tests will fail).
Barry
On Jul 19, 2023, at 11:40 AM, Satish Balay <[email protected] <mailto:[email protected]>> wrote:
BTW: Some explanation of configure:
It attempts the following on linux:
Source: #include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */ char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } <<<<<<<
Note - it does not include 'HYPRE.h' here - but redefines the prototype as 'char HYPRE_IJMatrixCreate();
Compiling it manually:
[balay@pj01 petsc]$ cat conftest.c char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } [balay@pj01 petsc]$ gcc -c conftest.c [balay@pj01 petsc]$ nm -Ao conftest.o |grep HYPRE_IJMatrixCreate conftest.o:0000000000000000 t _check_HYPRE_IJMatrixCreate conftest.o: U HYPRE_IJMatrixCreate [balay@pj01 petsc]$ nm -Ao arch-linux-c-debug/lib/libHYPRE.so |grep HYPRE_IJMatrixCreate arch-linux-c-debug/lib/libHYPRE.so:000000000007f2c2 T HYPRE_IJMatrixCreate [balay@pj01 petsc]$ <<<<
Here the "U HYPRE_IJMatrixCreate" in conftest.o matches "T HYPRE_IJMatrixCreate" in libHYPRE.so - so the "link" test in configure succeeds!
[balay@pj01 petsc]$ gcc -o conftest conftest.o arch-linux-c-debug/lib/libHYPRE.so [balay@pj01 petsc]$ echo $? 0 <<<<<
On windows - [due to name mangling by cdecl/stdcall, (/MT vs /MD) etc..] - this might not match - resulting in link failures.
Satish
On Wed, 19 Jul 2023, Satish Balay via petsc-users wrote:
You could try skipping this test [and assume --with-hypre-include and --with-hypre-lib options are correct] - and see if this works.
diff --git a/config/BuildSystem/config/packages/hypre.py b/config/BuildSystem/config/packages/hypre.py index 5bc88322aa2..2d6c7932e17 100644 --- a/config/BuildSystem/config/packages/hypre.py +++ b/config/BuildSystem/config/packages/hypre.py @@ -11,7 +11,7 @@ class Configure(config.package.GNUPackage): self.requiresversion = 1 self.gitcommit = 'v'+self.version self.download = ['git://https://github.com/hypre-space/hypre','https://github.com/hypre-space/hypre/archive/'+self.gitcommit+'.tar.gz'] - self.functions = ['HYPRE_IJMatrixCreate'] + self.functions = [] self.includes = ['HYPRE.h'] self.liblist = [['libHYPRE.a']] self.buildLanguages = ['C','Cxx']
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
You don't indicate what type of libraries you built hypre with; static or shared. My guess is you ended up with shared
I think the answer to your difficulty is hidden in __cdecl (Satish will know much better than me). When you are looking for symbols in Windows shared libraries you have to prepend something to the function prototype to have it successfully found. For example the PETSc include files have these things __declspec(dllimport) The configure test fails because it does not provide the needed prototype. Likely you built PTScotch with static libraries so no problem.
The simplest fix would be to build static hypre libraries. I think it is a major project to get PETSc configure and macro system to work properly with external packages that are in Windows shared libraries since more use of __declspec would be needed.
Barry
The PETSc installation instructions should probably say something about external packages with Windows shared libraries.
On Jul 19, 2023, at 10:52 AM, Daniel Stone <[email protected] <mailto:[email protected]>> wrote:
Hello,
I'm working on getting a petsc build running on windows. One necessary package to include is Hypre. I've been able to build Hypre seperately using cmake, and confirmed that the library works by setting up a VS project to run some of the example programs.
My attempted petsc build is being done through cygwin. I've been able to (with varying degrees of difficulty), build a fairly plain petsc, and one that downloads and builds ptscotch (after some modifications to both ptscotch and the config script). I am now attempting to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc). Note that the same compilers are being used for both Hypre and for petsc through cygwin - the new intel oneapi compilers (icx and ifx, after again varying amounts of pain to work around their awkwardness with the config script).
I'm seeing a problem when the config script does some tests on the included hypre lib. The source code looks like:
#include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */
#include "HYPRE.h"
char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main() { _check_HYPRE_IJMatrixCreate();; return 0; }
As I understand this is a fairly standard type of stub program used by the config script to check that it is able to link to certain symbols in given libraries. Tests like this have succeeded in my builds that include PTScotch.
I keep getting a linker error with the above test, including if I seperate it out and try to build it seperately:
unresolved external symbol "char __cdel HYPRE_IJMatrixCreate(void)" ....
Ok, it looks like a problem with either the library or linker commands. But here's the interesting thing - If I transplant this code into VS, with the same project setting that allows it to build the much more nontrivial Hypre example programs, I get the same error:
Error LNK2001 unresolved external symbol "char __cdecl HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1 C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1
So it seems like there might be something about this type of stub program that is not working with my Hypre library. I don't fully understand this program - it's able to call the function with no arguments, but it also needs to be linked against a library containing the function, apparently by wrapping it in a static void function? Not something I've seen before.
Does anyone have any insight into what might be going wrong - or really just any explaination of how the stub program works so I can figure out why it isn't in this case?
Many thanks,
Daniel
We don't have a good way to verify if the changes continue to work with the minver of the pkg [and if minor fixes can get this working - without increasing this requirement - sure without access to the new features provided by the new version] Having a single version dependency between pkgs makes the whole ecosystem [where many pkgs with all their dependencies mixed in] brittle.. Satish On Thu, 20 Jul 2023, Barry Smith wrote:
Satish,
I think hypre.py in main needs minversion = 2.29 instead of 2.14
On Jul 20, 2023, at 11:05 AM, Satish Balay <[email protected]> wrote:
Can check config/BuildSystem/config/packages/hypre.py
petsc-3.19 (or release branch) is compatible with hypre 2.28.0, petsc 'main' branch with 2.29.0
Satish
On Thu, 20 Jul 2023, Barry Smith via petsc-users wrote:
You cannot use this version of PETSc, 3.19, with the version of hypre you installed. In hypre they recently changed hypre_Error from an integer to a struct which completely breaks compatibility with previous versions of hypre (and hence previous versions of PETSc). You must use the main git branch of PETSc with the version of hypre you installed.
Barry
On Jul 20, 2023, at 5:10 AM, Daniel Stone <[email protected]> wrote:
Hi All,
Many thanks for the detailed explainations and ideas!
I tried skipping the test. When it came time to do the build itself (make $PETSC_DIR... all) I get some failures, unsurprisingly:
--------------------------------
FC arch-mswin-c-opt/obj/dm/f90-mod/petscdmplexmod.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-custom/zhypref.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-auto/hypref.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(444,29): error: assigning to 'hypre_Error' from incompatible type 'int' hypre__global_error = 0; ^ ~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7): note: expanded from macro 'PetscStackCallExternalVoid' __VA_ARGS__; \ ^~~~~~~~~~~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(634,29): error: assigning to 'hypre_Error' from incompatible type 'int' hypre__global_error = 0; ^ ~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7): note: expanded from macro 'PetscStackCallExternalVoid' __VA_ARGS__; \ ^~~~~~~~~~~ 2 errors generated. make[3]: *** [gmakefile:195: arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o] Error 1 make[3]: *** Waiting for unfinished jobs.... FC arch-mswin-c-opt/obj/ksp/f90-mod/petsckspdefmod.o CC arch-mswin-c-opt/obj/dm/impls/da/hypre/mhyp.o CC arch-mswin-c-opt/obj/mat/impls/hypre/mhypre.o make[3]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' make[2]: *** [/home/DanielOGS/petsc_ogs_3.19.1/lib/petsc/conf/rules.doc:28: libs] Error 2 make[2]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' **************************ERROR************************************* Error during compile, check arch-mswin-c-opt/lib/petsc/conf/make.log Send it and arch-mswin-c-opt/lib/petsc/conf/configure.log to [email protected] <mailto:[email protected]> ******************************************************************** Finishing make run at Wed, 19 Jul 2023 17:07:00 +0100
----------------------------------------
But wait - isn't this the compile stage, not the linking stage? This seems to imply that I've made a hash of providing include file such that a definition of "hypre_Error" cannot be seen - unless I'm misinterpreting. Interesting note about Hypre and include files - if built using configure and make, all the include files are conviniently copied into hypre/src/hypre/include. This is not done for a cmake build - I had to do the copying myself. Maybe I missed one.
On shared vs. static - if there a clear way of telling which I've ended up with? I've checked the cmakelists for hypre and this seems to imply that not-shared is the default, which I didn't change:
# Configuration options option(HYPRE_ENABLE_SHARED "Build a shared library" OFF) option(HYPRE_ENABLE_BIGINT "Use long long int for HYPRE_Int" OFF) option(HYPRE_ENABLE_MIXEDINT "Use long long int for HYPRE_BigInt, int for HYPRE_INT" OFF) [....]
checking again, I've noticed that the way that the stub-test fails is different depending on whether it's called from the config script or used in isolation - more details on that soon.
Thanks again,
Daniel
On Wed, Jul 19, 2023 at 4:58 PM Satish Balay via petsc-users <[email protected] <mailto:[email protected]>> wrote:
I think it should work with static libraries and 64bit compilers.
That's how I think --download-f2cblaslapack [etc] work.
Also it works with MS-MPI - even-though its a dll install, the library stub provides this symbol somehow..
balay@ps5 /cygdrive/c/Program Files (x86)/Microsoft SDKs/MPI/Lib/x64 $ nm -Ao msmpi.lib |grep " MPI_Init" msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init_thread msmpi.lib:msmpi.dll:0000000000000000 T MPI_Initialized
However - if the library symbol is somehow mangled - this configure mode of checking library functions will fail.
Checking PETSc dll build:
balay@ps5 ~/petsc/arch-ci-mswin-uni/lib $ nm -Ao libpetsc.lib |grep MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 I __imp_MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 T MatCreateSeqAIJWithArrays
It also has the unmangled symbol - so I guess this mode can work generally with dlls.
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
Satish,
So it will always fail on Windows with Windows compilers (both with static and shared libraries)? Is this true for all PETSc external packages? If so, why does the installation documentation say that some external packages can work with Windows compilers? (Presumably PETSc cannot since the configure tests will fail).
Barry
On Jul 19, 2023, at 11:40 AM, Satish Balay <[email protected] <mailto:[email protected]>> wrote:
BTW: Some explanation of configure:
It attempts the following on linux:
Source: #include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */ char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } <<<<<<<
Note - it does not include 'HYPRE.h' here - but redefines the prototype as 'char HYPRE_IJMatrixCreate();
Compiling it manually:
[balay@pj01 petsc]$ cat conftest.c char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } [balay@pj01 petsc]$ gcc -c conftest.c [balay@pj01 petsc]$ nm -Ao conftest.o |grep HYPRE_IJMatrixCreate conftest.o:0000000000000000 t _check_HYPRE_IJMatrixCreate conftest.o: U HYPRE_IJMatrixCreate [balay@pj01 petsc]$ nm -Ao arch-linux-c-debug/lib/libHYPRE.so |grep HYPRE_IJMatrixCreate arch-linux-c-debug/lib/libHYPRE.so:000000000007f2c2 T HYPRE_IJMatrixCreate [balay@pj01 petsc]$ <<<<
Here the "U HYPRE_IJMatrixCreate" in conftest.o matches "T HYPRE_IJMatrixCreate" in libHYPRE.so - so the "link" test in configure succeeds!
[balay@pj01 petsc]$ gcc -o conftest conftest.o arch-linux-c-debug/lib/libHYPRE.so [balay@pj01 petsc]$ echo $? 0 <<<<<
On windows - [due to name mangling by cdecl/stdcall, (/MT vs /MD) etc..] - this might not match - resulting in link failures.
Satish
On Wed, 19 Jul 2023, Satish Balay via petsc-users wrote:
You could try skipping this test [and assume --with-hypre-include and --with-hypre-lib options are correct] - and see if this works.
diff --git a/config/BuildSystem/config/packages/hypre.py b/config/BuildSystem/config/packages/hypre.py index 5bc88322aa2..2d6c7932e17 100644 --- a/config/BuildSystem/config/packages/hypre.py +++ b/config/BuildSystem/config/packages/hypre.py @@ -11,7 +11,7 @@ class Configure(config.package.GNUPackage): self.requiresversion = 1 self.gitcommit = 'v'+self.version self.download = ['git://https://github.com/hypre-space/hypre','https://github.com/hypre-space/hypre/archive/'+self.gitcommit+'.tar.gz'] - self.functions = ['HYPRE_IJMatrixCreate'] + self.functions = [] self.includes = ['HYPRE.h'] self.liblist = [['libHYPRE.a']] self.buildLanguages = ['C','Cxx']
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
You don't indicate what type of libraries you built hypre with; static or shared. My guess is you ended up with shared
I think the answer to your difficulty is hidden in __cdecl (Satish will know much better than me). When you are looking for symbols in Windows shared libraries you have to prepend something to the function prototype to have it successfully found. For example the PETSc include files have these things __declspec(dllimport) The configure test fails because it does not provide the needed prototype. Likely you built PTScotch with static libraries so no problem.
The simplest fix would be to build static hypre libraries. I think it is a major project to get PETSc configure and macro system to work properly with external packages that are in Windows shared libraries since more use of __declspec would be needed.
Barry
The PETSc installation instructions should probably say something about external packages with Windows shared libraries.
On Jul 19, 2023, at 10:52 AM, Daniel Stone <[email protected] <mailto:[email protected]>> wrote:
Hello,
I'm working on getting a petsc build running on windows. One necessary package to include is Hypre. I've been able to build Hypre seperately using cmake, and confirmed that the library works by setting up a VS project to run some of the example programs.
My attempted petsc build is being done through cygwin. I've been able to (with varying degrees of difficulty), build a fairly plain petsc, and one that downloads and builds ptscotch (after some modifications to both ptscotch and the config script). I am now attempting to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc). Note that the same compilers are being used for both Hypre and for petsc through cygwin - the new intel oneapi compilers (icx and ifx, after again varying amounts of pain to work around their awkwardness with the config script).
I'm seeing a problem when the config script does some tests on the included hypre lib. The source code looks like:
#include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */
#include "HYPRE.h"
char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main() { _check_HYPRE_IJMatrixCreate();; return 0; }
As I understand this is a fairly standard type of stub program used by the config script to check that it is able to link to certain symbols in given libraries. Tests like this have succeeded in my builds that include PTScotch.
I keep getting a linker error with the above test, including if I seperate it out and try to build it seperately:
unresolved external symbol "char __cdel HYPRE_IJMatrixCreate(void)" ....
Ok, it looks like a problem with either the library or linker commands. But here's the interesting thing - If I transplant this code into VS, with the same project setting that allows it to build the much more nontrivial Hypre example programs, I get the same error:
Error LNK2001 unresolved external symbol "char __cdecl HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1 C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1
So it seems like there might be something about this type of stub program that is not working with my Hypre library. I don't fully understand this program - it's able to call the function with no arguments, but it also needs to be linked against a library containing the function, apparently by wrapping it in a static void function? Not something I've seen before.
Does anyone have any insight into what might be going wrong - or really just any explaination of how the stub program works so I can figure out why it isn't in this case?
Many thanks,
Daniel
hypre_Error was changed from an integer to a struct in hypre. PETSc code was changed in main to work with that struct and not work if hypre_Error is an int. This means main cannot work with previous hypre versions (where hypre_Error is an int). Sure, instead of changing the minimum version one could potentially change PETSc's main version source code to work with both old and new hypre, but that would need to be done and has not been done. main is simply broken, it allows configure to succeed with an older version of hypre but PETSc main cannot compile with this, thus confusing developers and users. This does need to be fixed, one way or another.
On Jul 21, 2023, at 1:32 PM, Satish Balay <[email protected]> wrote:
We don't have a good way to verify if the changes continue to work with the minver of the pkg [and if minor fixes can get this working - without increasing this requirement - sure without access to the new features provided by the new version]
Having a single version dependency between pkgs makes the whole ecosystem [where many pkgs with all their dependencies mixed in] brittle..
Satish
On Thu, 20 Jul 2023, Barry Smith wrote:
Satish,
I think hypre.py in main needs minversion = 2.29 instead of 2.14
On Jul 20, 2023, at 11:05 AM, Satish Balay <[email protected]> wrote:
Can check config/BuildSystem/config/packages/hypre.py
petsc-3.19 (or release branch) is compatible with hypre 2.28.0, petsc 'main' branch with 2.29.0
Satish
On Thu, 20 Jul 2023, Barry Smith via petsc-users wrote:
You cannot use this version of PETSc, 3.19, with the version of hypre you installed. In hypre they recently changed hypre_Error from an integer to a struct which completely breaks compatibility with previous versions of hypre (and hence previous versions of PETSc). You must use the main git branch of PETSc with the version of hypre you installed.
Barry
On Jul 20, 2023, at 5:10 AM, Daniel Stone <[email protected]> wrote:
Hi All,
Many thanks for the detailed explainations and ideas!
I tried skipping the test. When it came time to do the build itself (make $PETSC_DIR... all) I get some failures, unsurprisingly:
--------------------------------
FC arch-mswin-c-opt/obj/dm/f90-mod/petscdmplexmod.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-custom/zhypref.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-auto/hypref.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(444,29): error: assigning to 'hypre_Error' from incompatible type 'int' hypre__global_error = 0; ^ ~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7): note: expanded from macro 'PetscStackCallExternalVoid' __VA_ARGS__; \ ^~~~~~~~~~~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(634,29): error: assigning to 'hypre_Error' from incompatible type 'int' hypre__global_error = 0; ^ ~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7): note: expanded from macro 'PetscStackCallExternalVoid' __VA_ARGS__; \ ^~~~~~~~~~~ 2 errors generated. make[3]: *** [gmakefile:195: arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o] Error 1 make[3]: *** Waiting for unfinished jobs.... FC arch-mswin-c-opt/obj/ksp/f90-mod/petsckspdefmod.o CC arch-mswin-c-opt/obj/dm/impls/da/hypre/mhyp.o CC arch-mswin-c-opt/obj/mat/impls/hypre/mhypre.o make[3]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' make[2]: *** [/home/DanielOGS/petsc_ogs_3.19.1/lib/petsc/conf/rules.doc:28: libs] Error 2 make[2]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' **************************ERROR************************************* Error during compile, check arch-mswin-c-opt/lib/petsc/conf/make.log Send it and arch-mswin-c-opt/lib/petsc/conf/configure.log to [email protected] <mailto:[email protected]> ******************************************************************** Finishing make run at Wed, 19 Jul 2023 17:07:00 +0100
----------------------------------------
But wait - isn't this the compile stage, not the linking stage? This seems to imply that I've made a hash of providing include file such that a definition of "hypre_Error" cannot be seen - unless I'm misinterpreting. Interesting note about Hypre and include files - if built using configure and make, all the include files are conviniently copied into hypre/src/hypre/include. This is not done for a cmake build - I had to do the copying myself. Maybe I missed one.
On shared vs. static - if there a clear way of telling which I've ended up with? I've checked the cmakelists for hypre and this seems to imply that not-shared is the default, which I didn't change:
# Configuration options option(HYPRE_ENABLE_SHARED "Build a shared library" OFF) option(HYPRE_ENABLE_BIGINT "Use long long int for HYPRE_Int" OFF) option(HYPRE_ENABLE_MIXEDINT "Use long long int for HYPRE_BigInt, int for HYPRE_INT" OFF) [....]
checking again, I've noticed that the way that the stub-test fails is different depending on whether it's called from the config script or used in isolation - more details on that soon.
Thanks again,
Daniel
On Wed, Jul 19, 2023 at 4:58 PM Satish Balay via petsc-users <[email protected] <mailto:[email protected]>> wrote:
I think it should work with static libraries and 64bit compilers.
That's how I think --download-f2cblaslapack [etc] work.
Also it works with MS-MPI - even-though its a dll install, the library stub provides this symbol somehow..
balay@ps5 /cygdrive/c/Program Files (x86)/Microsoft SDKs/MPI/Lib/x64 $ nm -Ao msmpi.lib |grep " MPI_Init" msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init_thread msmpi.lib:msmpi.dll:0000000000000000 T MPI_Initialized
However - if the library symbol is somehow mangled - this configure mode of checking library functions will fail.
Checking PETSc dll build:
balay@ps5 ~/petsc/arch-ci-mswin-uni/lib $ nm -Ao libpetsc.lib |grep MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 I __imp_MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 T MatCreateSeqAIJWithArrays
It also has the unmangled symbol - so I guess this mode can work generally with dlls.
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
Satish,
So it will always fail on Windows with Windows compilers (both with static and shared libraries)? Is this true for all PETSc external packages? If so, why does the installation documentation say that some external packages can work with Windows compilers? (Presumably PETSc cannot since the configure tests will fail).
Barry
On Jul 19, 2023, at 11:40 AM, Satish Balay <[email protected] <mailto:[email protected]>> wrote:
BTW: Some explanation of configure:
It attempts the following on linux:
Source: #include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */ char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } <<<<<<<
Note - it does not include 'HYPRE.h' here - but redefines the prototype as 'char HYPRE_IJMatrixCreate();
Compiling it manually:
[balay@pj01 petsc]$ cat conftest.c char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } [balay@pj01 petsc]$ gcc -c conftest.c [balay@pj01 petsc]$ nm -Ao conftest.o |grep HYPRE_IJMatrixCreate conftest.o:0000000000000000 t _check_HYPRE_IJMatrixCreate conftest.o: U HYPRE_IJMatrixCreate [balay@pj01 petsc]$ nm -Ao arch-linux-c-debug/lib/libHYPRE.so |grep HYPRE_IJMatrixCreate arch-linux-c-debug/lib/libHYPRE.so:000000000007f2c2 T HYPRE_IJMatrixCreate [balay@pj01 petsc]$ <<<<
Here the "U HYPRE_IJMatrixCreate" in conftest.o matches "T HYPRE_IJMatrixCreate" in libHYPRE.so - so the "link" test in configure succeeds!
[balay@pj01 petsc]$ gcc -o conftest conftest.o arch-linux-c-debug/lib/libHYPRE.so [balay@pj01 petsc]$ echo $? 0 <<<<<
On windows - [due to name mangling by cdecl/stdcall, (/MT vs /MD) etc..] - this might not match - resulting in link failures.
Satish
On Wed, 19 Jul 2023, Satish Balay via petsc-users wrote:
You could try skipping this test [and assume --with-hypre-include and --with-hypre-lib options are correct] - and see if this works.
diff --git a/config/BuildSystem/config/packages/hypre.py b/config/BuildSystem/config/packages/hypre.py index 5bc88322aa2..2d6c7932e17 100644 --- a/config/BuildSystem/config/packages/hypre.py +++ b/config/BuildSystem/config/packages/hypre.py @@ -11,7 +11,7 @@ class Configure(config.package.GNUPackage): self.requiresversion = 1 self.gitcommit = 'v'+self.version self.download = ['git://https://github.com/hypre-space/hypre','https://github.com/hypre-space/hypre/archive/'+self.gitcommit+'.tar.gz'] - self.functions = ['HYPRE_IJMatrixCreate'] + self.functions = [] self.includes = ['HYPRE.h'] self.liblist = [['libHYPRE.a']] self.buildLanguages = ['C','Cxx']
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
You don't indicate what type of libraries you built hypre with; static or shared. My guess is you ended up with shared
I think the answer to your difficulty is hidden in __cdecl (Satish will know much better than me). When you are looking for symbols in Windows shared libraries you have to prepend something to the function prototype to have it successfully found. For example the PETSc include files have these things __declspec(dllimport) The configure test fails because it does not provide the needed prototype. Likely you built PTScotch with static libraries so no problem.
The simplest fix would be to build static hypre libraries. I think it is a major project to get PETSc configure and macro system to work properly with external packages that are in Windows shared libraries since more use of __declspec would be needed.
Barry
The PETSc installation instructions should probably say something about external packages with Windows shared libraries.
On Jul 19, 2023, at 10:52 AM, Daniel Stone <[email protected] <mailto:[email protected]>> wrote:
Hello,
I'm working on getting a petsc build running on windows. One necessary package to include is Hypre. I've been able to build Hypre seperately using cmake, and confirmed that the library works by setting up a VS project to run some of the example programs.
My attempted petsc build is being done through cygwin. I've been able to (with varying degrees of difficulty), build a fairly plain petsc, and one that downloads and builds ptscotch (after some modifications to both ptscotch and the config script). I am now attempting to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc). Note that the same compilers are being used for both Hypre and for petsc through cygwin - the new intel oneapi compilers (icx and ifx, after again varying amounts of pain to work around their awkwardness with the config script).
I'm seeing a problem when the config script does some tests on the included hypre lib. The source code looks like:
#include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */
#include "HYPRE.h"
char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main() { _check_HYPRE_IJMatrixCreate();; return 0; }
As I understand this is a fairly standard type of stub program used by the config script to check that it is able to link to certain symbols in given libraries. Tests like this have succeeded in my builds that include PTScotch.
I keep getting a linker error with the above test, including if I seperate it out and try to build it seperately:
unresolved external symbol "char __cdel HYPRE_IJMatrixCreate(void)" ....
Ok, it looks like a problem with either the library or linker commands. But here's the interesting thing - If I transplant this code into VS, with the same project setting that allows it to build the much more nontrivial Hypre example programs, I get the same error:
Error LNK2001 unresolved external symbol "char __cdecl HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1 C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1
So it seems like there might be something about this type of stub program that is not working with my Hypre library. I don't fully understand this program - it's able to call the function with no arguments, but it also needs to be linked against a library containing the function, apparently by wrapping it in a static void function? Not something I've seen before.
Does anyone have any insight into what might be going wrong - or really just any explaination of how the stub program works so I can figure out why it isn't in this case?
Many thanks,
Daniel
On the hypre versioning - aha. For this project I locked the petsc version a little while ago (3.19.1), but I've been using a fresh clone of hypre, so clearly it's too modern a version. Using the appropriate version of hypre (2.28.0, according to hypre.py) might fix some things. I may have other problems in the form of the confilicting compiler options as Satish mentioned, which I'll have to figure out too. Thanks, Daniel On Fri, Jul 21, 2023 at 9:32 PM Barry Smith <[email protected]> wrote:
hypre_Error was changed from an integer to a struct in hypre. PETSc code was changed in main to work with that struct and not work if hypre_Error is an int. This means main cannot work with previous hypre versions (where hypre_Error is an int). Sure, instead of changing the minimum version one could potentially change PETSc's main version source code to work with both old and new hypre, but that would need to be done and has not been done. main is simply broken, it allows configure to succeed with an older version of hypre but PETSc main cannot compile with this, thus confusing developers and users. This does need to be fixed, one way or another.
On Jul 21, 2023, at 1:32 PM, Satish Balay <[email protected]> wrote:
We don't have a good way to verify if the changes continue to work with the minver of the pkg [and if minor fixes can get this working - without increasing this requirement - sure without access to the new features provided by the new version]
Having a single version dependency between pkgs makes the whole ecosystem [where many pkgs with all their dependencies mixed in] brittle..
Satish
On Thu, 20 Jul 2023, Barry Smith wrote:
Satish,
I think hypre.py in main needs minversion = 2.29 instead of 2.14
On Jul 20, 2023, at 11:05 AM, Satish Balay <[email protected]> wrote:
Can check config/BuildSystem/config/packages/hypre.py
petsc-3.19 (or release branch) is compatible with hypre 2.28.0, petsc
'main' branch with 2.29.0
Satish
On Thu, 20 Jul 2023, Barry Smith via petsc-users wrote:
You cannot use this version of PETSc, 3.19, with the version of hypre
you installed. In hypre they recently changed hypre_Error from an integer to a struct which completely breaks compatibility with previous versions of hypre (and hence previous versions of PETSc). You must use the main git branch of PETSc with the version of hypre you installed.
Barry
On Jul 20, 2023, at 5:10 AM, Daniel Stone <
[email protected]> wrote:
Hi All,
Many thanks for the detailed explainations and ideas!
I tried skipping the test. When it came time to do the build itself
(make $PETSC_DIR... all) I get some failures, unsurprisingly:
--------------------------------
FC arch-mswin-c-opt/obj/dm/f90-mod/petscdmplexmod.o CC
arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-custom/zhypref.o
CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-auto/hypref.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o
C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(444,29): error: assigning to 'hypre_Error' from incompatible type 'int'
hypre__global_error = 0; ^ ~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7):
note: expanded from macro 'PetscStackCallExternalVoid'
__VA_ARGS__; \ ^~~~~~~~~~~
C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(634,29): error: assigning to 'hypre_Error' from incompatible type 'int'
hypre__global_error = 0; ^ ~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7):
note: expanded from macro 'PetscStackCallExternalVoid'
__VA_ARGS__; \ ^~~~~~~~~~~ 2 errors generated. make[3]: *** [gmakefile:195:
arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o] Error 1
make[3]: *** Waiting for unfinished jobs.... FC arch-mswin-c-opt/obj/ksp/f90-mod/petsckspdefmod.o CC arch-mswin-c-opt/obj/dm/impls/da/hypre/mhyp.o CC arch-mswin-c-opt/obj/mat/impls/hypre/mhypre.o make[3]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' make[2]: *** [/home/DanielOGS/petsc_ogs_3.19.1/lib/petsc/conf/rules.doc:28: libs] Error 2 make[2]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' **************************ERROR************************************* Error during compile, check arch-mswin-c-opt/lib/petsc/conf/make.log Send it and arch-mswin-c-opt/lib/petsc/conf/configure.log to [email protected] <mailto:[email protected]> ******************************************************************** Finishing make run at Wed, 19 Jul 2023 17:07:00 +0100
----------------------------------------
But wait - isn't this the compile stage, not the linking stage? This seems to imply that I've made a hash of providing include file such that a definition of "hypre_Error" cannot be seen - unless I'm misinterpreting. Interesting note about Hypre and include files - if built using configure and make, all the include files are conviniently copied into hypre/src/hypre/include. This is not done for a cmake build - I had to do the copying myself. Maybe I missed one.
On shared vs. static - if there a clear way of telling which I've ended up with? I've checked the cmakelists for hypre and this seems to imply that not-shared is the default, which I didn't change:
# Configuration options option(HYPRE_ENABLE_SHARED "Build a shared library" OFF) option(HYPRE_ENABLE_BIGINT "Use long long int for HYPRE_Int" OFF) option(HYPRE_ENABLE_MIXEDINT "Use long long int for HYPRE_BigInt, int for HYPRE_INT" OFF) [....]
checking again, I've noticed that the way that the stub-test fails is different depending on whether it's called from the config script or used in isolation - more details on that soon.
Thanks again,
Daniel
On Wed, Jul 19, 2023 at 4:58 PM Satish Balay via petsc-users < [email protected] <mailto:[email protected]>> wrote:
I think it should work with static libraries and 64bit compilers.
That's how I think --download-f2cblaslapack [etc] work.
Also it works with MS-MPI - even-though its a dll install, the library stub provides this symbol somehow..
balay@ps5 /cygdrive/c/Program Files (x86)/Microsoft SDKs/MPI/Lib/x64 $ nm -Ao msmpi.lib |grep " MPI_Init" msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init_thread msmpi.lib:msmpi.dll:0000000000000000 T MPI_Initialized
However - if the library symbol is somehow mangled - this configure mode of checking library functions will fail.
Checking PETSc dll build:
balay@ps5 ~/petsc/arch-ci-mswin-uni/lib $ nm -Ao libpetsc.lib |grep MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 I __imp_MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 T MatCreateSeqAIJWithArrays
It also has the unmangled symbol - so I guess this mode can work generally with dlls.
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
Satish,
So it will always fail on Windows with Windows compilers (both
with static and shared libraries)? Is this true for all PETSc external packages? If so, why does the installation documentation say that some external packages can work with Windows compilers? (Presumably PETSc cannot since the configure tests will fail).
Barry
On Jul 19, 2023, at 11:40 AM, Satish Balay <[email protected]
<mailto:[email protected]>> wrote:
BTW: Some explanation of configure:
It attempts the following on linux:
Source: #include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */ char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() {
HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } <<<<<<<
Note - it does not include 'HYPRE.h' here - but redefines the
prototype as 'char HYPRE_IJMatrixCreate();
Compiling it manually:
[balay@pj01 petsc]$ cat conftest.c char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() {
HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } [balay@pj01 petsc]$ gcc -c conftest.c [balay@pj01 petsc]$ nm -Ao conftest.o |grep HYPRE_IJMatrixCreate conftest.o:0000000000000000 t _check_HYPRE_IJMatrixCreate conftest.o: U HYPRE_IJMatrixCreate [balay@pj01 petsc]$ nm -Ao arch-linux-c-debug/lib/libHYPRE.so
|grep HYPRE_IJMatrixCreate
arch-linux-c-debug/lib/libHYPRE.so:000000000007f2c2 T HYPRE_IJMatrixCreate [balay@pj01 petsc]$ <<<<
Here the "U HYPRE_IJMatrixCreate" in conftest.o matches "T HYPRE_IJMatrixCreate" in libHYPRE.so - so the "link" test in configure succeeds!
[balay@pj01 petsc]$ gcc -o conftest conftest.o arch-linux-c-debug/lib/libHYPRE.so [balay@pj01 petsc]$ echo $? 0 <<<<<
On windows - [due to name mangling by cdecl/stdcall, (/MT vs /MD) etc..] - this might not match - resulting in link failures.
Satish
On Wed, 19 Jul 2023, Satish Balay via petsc-users wrote:
You could try skipping this test [and assume --with-hypre-include and --with-hypre-lib options are correct] - and see if this works.
diff --git a/config/BuildSystem/config/packages/hypre.py b/config/BuildSystem/config/packages/hypre.py index 5bc88322aa2..2d6c7932e17 100644 --- a/config/BuildSystem/config/packages/hypre.py +++ b/config/BuildSystem/config/packages/hypre.py @@ -11,7 +11,7 @@ class Configure(config.package.GNUPackage): self.requiresversion = 1 self.gitcommit = 'v'+self.version self.download = ['git:// https://github.com/hypre-space/hypre',' https://github.com/hypre-space/hypre/archive/'+self.gitcommit+'.tar.gz'] - self.functions = ['HYPRE_IJMatrixCreate'] + self.functions = [] self.includes = ['HYPRE.h'] self.liblist = [['libHYPRE.a']] self.buildLanguages = ['C','Cxx']
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
You don't indicate what type of libraries you built hypre with;
static or shared. My guess is you ended up with shared
I think the answer to your difficulty is hidden in __cdecl
(Satish will know much better than me). When you are looking for symbols in Windows shared libraries you have to prepend something to the function prototype to have it successfully found. For example the PETSc include files have these things __declspec(dllimport) The configure test fails because it does not provide the needed prototype. Likely you built PTScotch with static libraries so no problem.
The simplest fix would be to build static hypre libraries. I
think it is a major project to get PETSc configure and macro system to work properly with external packages that are in Windows shared libraries since more use of __declspec would be needed.
Barry
The PETSc installation instructions should probably say
something about external packages with Windows shared libraries.
On Jul 19, 2023, at 10:52 AM, Daniel Stone <
[email protected] <mailto:[email protected]>> wrote:
Hello,
I'm working on getting a petsc build running on windows. One
necessary package to include is Hypre. I've been able to build Hypre seperately using cmake, and confirmed that the library works
by setting up a VS project to run some of the example programs.
My attempted petsc build is being done through cygwin. I've been able to (with varying degrees of difficulty), build a fairly plain petsc, and one that downloads and builds ptscotch (after some modifications to both ptscotch and the config script). I am now attempting to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc). Note that the same compilers are being used for both Hypre and for petsc through cygwin - the new intel oneapi compilers (icx and ifx, after again varying amounts of pain to work around their awkwardness with the config script).
I'm seeing a problem when the config script does some tests on the included hypre lib. The source code looks like:
#include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */
#include "HYPRE.h"
char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main() { _check_HYPRE_IJMatrixCreate();; return 0; }
As I understand this is a fairly standard type of stub program used by the config script to check that it is able to link to certain symbols in given libraries. Tests like this have succeeded in my builds that include PTScotch.
I keep getting a linker error with the above test, including if I seperate it out and try to build it seperately:
unresolved external symbol "char __cdel HYPRE_IJMatrixCreate(void)" ....
Ok, it looks like a problem with either the library or linker commands. But here's the interesting thing - If I transplant this code into VS, with the same project setting that allows it to build the much more nontrivial Hypre example programs, I get the same error:
Error LNK2001 unresolved external symbol "char __cdecl HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1 C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1
So it seems like there might be something about this type of stub program that is not working with my Hypre library. I don't fully understand this program - it's able to call the function with no arguments, but it also needs to be linked against a library containing the function, apparently by wrapping it in a static void function? Not something I've seen before.
Does anyone have any insight into what might be going wrong - or really just any explaination of how the stub program works so I can figure out why it isn't in this case?
Many thanks,
Daniel
This still isn't working, and it looks like the md/mt distinciton might be the culprit. I use the Ninja generator with cmake, on the intel oneapi command line (i.e. windows command line but with the pathing and other things sorted out for the new intel compilers). Looking at the build.ninja file produced, I see rules like: ------------------- build CMakeFiles\HYPRE.dir\blas\dasum.c.obj: C_COMPILER__HYPRE_Debug ..\blas\dasum.c || cmake_object_order_depends_target_HYPRE DEFINES = -D_CRT_SECURE_NO_WARNINGS DEP_FILE = CMakeFiles\HYPRE.dir\blas\dasum.c.obj.d FLAGS = /DWIN32 /D_WINDOWS /Zi /Ob0 /Od /RTC1 -MDd INCLUDES = [etc....] ------------------ Note the -MDd flag (which is an -MD flag if I ask for a non debug build). It's odd that it isn't an /MD flag, but I've done some quick checks on the oneapi command line and it looks like -MD is still interpreted as valid input by icx (e.g., >> icx /MD and >> icx -MD both only complain about lack of input files, while >> icx /MDblahblahnotarealflag and >> icx -MDblahblahnotarealflag both give "unknown argument" errors). Thus it really seems like my Hypre library has been building with the MD option, although MT is the default ( https://www.intel.com/content/www/us/en/docs/dpcpp-cpp-compiler/developer-gu... and https://www.intel.com/content/www/us/en/docs/dpcpp-cpp-compiler/developer-gu...), as suggested by Satish. There seems to be no hint of the MD flag in any of the cmake files provided with hypre, so it looks like the -MD flag is being added by the cmake Ninja generator as a kind of default (and maybe in a confused way - why the - instead of /?) So the task now seems to be to convince cmake to stop doing this. On Mon, Jul 24, 2023 at 11:21 AM Daniel Stone <[email protected]> wrote:
On the hypre versioning - aha. For this project I locked the petsc version a little while ago (3.19.1), but I've been using a fresh clone of hypre, so clearly it's too modern a version. Using the appropriate version of hypre (2.28.0, according to hypre.py) might fix some things. I may have other problems in the form of the confilicting compiler options as Satish mentioned, which I'll have to figure out too.
Thanks,
Daniel
On Fri, Jul 21, 2023 at 9:32 PM Barry Smith <[email protected]> wrote:
hypre_Error was changed from an integer to a struct in hypre. PETSc code was changed in main to work with that struct and not work if hypre_Error is an int. This means main cannot work with previous hypre versions (where hypre_Error is an int). Sure, instead of changing the minimum version one could potentially change PETSc's main version source code to work with both old and new hypre, but that would need to be done and has not been done. main is simply broken, it allows configure to succeed with an older version of hypre but PETSc main cannot compile with this, thus confusing developers and users. This does need to be fixed, one way or another.
On Jul 21, 2023, at 1:32 PM, Satish Balay <[email protected]> wrote:
We don't have a good way to verify if the changes continue to work with the minver of the pkg [and if minor fixes can get this working - without increasing this requirement - sure without access to the new features provided by the new version]
Having a single version dependency between pkgs makes the whole ecosystem [where many pkgs with all their dependencies mixed in] brittle..
Satish
On Thu, 20 Jul 2023, Barry Smith wrote:
Satish,
I think hypre.py in main needs minversion = 2.29 instead of 2.14
On Jul 20, 2023, at 11:05 AM, Satish Balay <[email protected]> wrote:
Can check config/BuildSystem/config/packages/hypre.py
petsc-3.19 (or release branch) is compatible with hypre 2.28.0, petsc
'main' branch with 2.29.0
Satish
On Thu, 20 Jul 2023, Barry Smith via petsc-users wrote:
You cannot use this version of PETSc, 3.19, with the version of
hypre you installed. In hypre they recently changed hypre_Error from an integer to a struct which completely breaks compatibility with previous versions of hypre (and hence previous versions of PETSc). You must use the main git branch of PETSc with the version of hypre you installed.
Barry
On Jul 20, 2023, at 5:10 AM, Daniel Stone <
[email protected]> wrote:
Hi All,
Many thanks for the detailed explainations and ideas!
I tried skipping the test. When it came time to do the build itself
(make $PETSC_DIR... all) I get some failures, unsurprisingly:
--------------------------------
FC arch-mswin-c-opt/obj/dm/f90-mod/petscdmplexmod.o CC
arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-custom/zhypref.o
CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/ftn-auto/hypref.o CC arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o
C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(444,29): error: assigning to 'hypre_Error' from incompatible type 'int'
hypre__global_error = 0; ^ ~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7):
note: expanded from macro 'PetscStackCallExternalVoid'
__VA_ARGS__; \ ^~~~~~~~~~~
C:\cygwin64\home\DANIEL~1\PETSC_~1.1\src\ksp\pc\impls\hypre\hypre.c(634,29): error: assigning to 'hypre_Error' from incompatible type 'int'
hypre__global_error = 0; ^ ~ C:\cygwin64\home\DANIEL~1\PETSC_~1.1\include\petscerror.h(1752,7):
note: expanded from macro 'PetscStackCallExternalVoid'
__VA_ARGS__; \ ^~~~~~~~~~~ 2 errors generated. make[3]: *** [gmakefile:195:
arch-mswin-c-opt/obj/ksp/pc/impls/hypre/hypre.o] Error 1
make[3]: *** Waiting for unfinished jobs.... FC arch-mswin-c-opt/obj/ksp/f90-mod/petsckspdefmod.o CC arch-mswin-c-opt/obj/dm/impls/da/hypre/mhyp.o CC arch-mswin-c-opt/obj/mat/impls/hypre/mhypre.o make[3]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' make[2]: *** [/home/DanielOGS/petsc_ogs_3.19.1/lib/petsc/conf/rules.doc:28: libs] Error 2 make[2]: Leaving directory '/home/DanielOGS/petsc_ogs_3.19.1' **************************ERROR************************************* Error during compile, check arch-mswin-c-opt/lib/petsc/conf/make.log Send it and arch-mswin-c-opt/lib/petsc/conf/configure.log to [email protected] <mailto:[email protected]> ******************************************************************** Finishing make run at Wed, 19 Jul 2023 17:07:00 +0100
----------------------------------------
But wait - isn't this the compile stage, not the linking stage? This seems to imply that I've made a hash of providing include file such that a definition of "hypre_Error" cannot be seen - unless I'm misinterpreting. Interesting note about Hypre and include files - if built using configure and make, all the include files are conviniently copied into hypre/src/hypre/include. This is not done for a cmake build - I had to do the copying myself. Maybe I missed one.
On shared vs. static - if there a clear way of telling which I've ended up with? I've checked the cmakelists for hypre and this seems to imply that not-shared is the default, which I didn't change:
# Configuration options option(HYPRE_ENABLE_SHARED "Build a shared library" OFF) option(HYPRE_ENABLE_BIGINT "Use long long int for HYPRE_Int" OFF) option(HYPRE_ENABLE_MIXEDINT "Use long long int for HYPRE_BigInt, int for HYPRE_INT" OFF) [....]
checking again, I've noticed that the way that the stub-test fails is different depending on whether it's called from the config script or used in isolation - more details on that soon.
Thanks again,
Daniel
On Wed, Jul 19, 2023 at 4:58 PM Satish Balay via petsc-users < [email protected] <mailto:[email protected]>> wrote:
I think it should work with static libraries and 64bit compilers.
That's how I think --download-f2cblaslapack [etc] work.
Also it works with MS-MPI - even-though its a dll install, the library stub provides this symbol somehow..
balay@ps5 /cygdrive/c/Program Files (x86)/Microsoft SDKs/MPI/Lib/x64 $ nm -Ao msmpi.lib |grep " MPI_Init" msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init_thread msmpi.lib:msmpi.dll:0000000000000000 T MPI_Initialized
However - if the library symbol is somehow mangled - this configure mode of checking library functions will fail.
Checking PETSc dll build:
balay@ps5 ~/petsc/arch-ci-mswin-uni/lib $ nm -Ao libpetsc.lib |grep MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 I __imp_MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 T MatCreateSeqAIJWithArrays
It also has the unmangled symbol - so I guess this mode can work generally with dlls.
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
Satish,
So it will always fail on Windows with Windows compilers (both
with static and shared libraries)? Is this true for all PETSc external packages? If so, why does the installation documentation say that some external packages can work with Windows compilers? (Presumably PETSc cannot since the configure tests will fail).
Barry
On Jul 19, 2023, at 11:40 AM, Satish Balay <[email protected]
<mailto:[email protected]>> wrote:
BTW: Some explanation of configure:
It attempts the following on linux:
Source: #include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */ char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() {
HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } <<<<<<<
Note - it does not include 'HYPRE.h' here - but redefines the
prototype as 'char HYPRE_IJMatrixCreate();
Compiling it manually:
[balay@pj01 petsc]$ cat conftest.c char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() {
HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } [balay@pj01 petsc]$ gcc -c conftest.c [balay@pj01 petsc]$ nm -Ao conftest.o |grep HYPRE_IJMatrixCreate conftest.o:0000000000000000 t _check_HYPRE_IJMatrixCreate conftest.o: U HYPRE_IJMatrixCreate [balay@pj01 petsc]$ nm -Ao arch-linux-c-debug/lib/libHYPRE.so
|grep HYPRE_IJMatrixCreate
arch-linux-c-debug/lib/libHYPRE.so:000000000007f2c2 T HYPRE_IJMatrixCreate [balay@pj01 petsc]$ <<<<
Here the "U HYPRE_IJMatrixCreate" in conftest.o matches "T HYPRE_IJMatrixCreate" in libHYPRE.so - so the "link" test in configure succeeds!
[balay@pj01 petsc]$ gcc -o conftest conftest.o arch-linux-c-debug/lib/libHYPRE.so [balay@pj01 petsc]$ echo $? 0 <<<<<
On windows - [due to name mangling by cdecl/stdcall, (/MT vs /MD) etc..] - this might not match - resulting in link failures.
Satish
On Wed, 19 Jul 2023, Satish Balay via petsc-users wrote:
You could try skipping this test [and assume --with-hypre-include and --with-hypre-lib options are correct] - and see if this works.
diff --git a/config/BuildSystem/config/packages/hypre.py b/config/BuildSystem/config/packages/hypre.py index 5bc88322aa2..2d6c7932e17 100644 --- a/config/BuildSystem/config/packages/hypre.py +++ b/config/BuildSystem/config/packages/hypre.py @@ -11,7 +11,7 @@ class Configure(config.package.GNUPackage): self.requiresversion = 1 self.gitcommit = 'v'+self.version self.download = ['git:// https://github.com/hypre-space/hypre',' https://github.com/hypre-space/hypre/archive/'+self.gitcommit+'.tar.gz'] - self.functions = ['HYPRE_IJMatrixCreate'] + self.functions = [] self.includes = ['HYPRE.h'] self.liblist = [['libHYPRE.a']] self.buildLanguages = ['C','Cxx']
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
You don't indicate what type of libraries you built hypre
with; static or shared. My guess is you ended up with shared
I think the answer to your difficulty is hidden in __cdecl
(Satish will know much better than me). When you are looking for symbols in Windows shared libraries you have to prepend something to the function prototype to have it successfully found. For example the PETSc include files have these things __declspec(dllimport) The configure test fails because it does not provide the needed prototype. Likely you built PTScotch with static libraries so no problem.
The simplest fix would be to build static hypre libraries. I
think it is a major project to get PETSc configure and macro system to work properly with external packages that are in Windows shared libraries since more use of __declspec would be needed.
Barry
The PETSc installation instructions should probably say
something about external packages with Windows shared libraries.
On Jul 19, 2023, at 10:52 AM, Daniel Stone <
[email protected] <mailto:[email protected]>> wrote:
Hello,
I'm working on getting a petsc build running on windows. One
necessary package to include is Hypre. I've been able to build Hypre seperately using cmake, and confirmed that the library works
by setting up a VS project to run some of the example programs.
My attempted petsc build is being done through cygwin. I've been able to (with varying degrees of difficulty), build a fairly plain petsc, and one that downloads and builds ptscotch (after some modifications to both ptscotch and the config script). I am now attempting to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc). Note that the same compilers are being used for both Hypre and for petsc through cygwin - the new intel oneapi compilers (icx and ifx, after again varying amounts of pain to work around their awkwardness with the config script).
I'm seeing a problem when the config script does some tests on the included hypre lib. The source code looks like:
#include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */
#include "HYPRE.h"
char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main() { _check_HYPRE_IJMatrixCreate();; return 0; }
As I understand this is a fairly standard type of stub program used by the config script to check that it is able to link to certain symbols in given libraries. Tests like this have succeeded in my builds that include PTScotch.
I keep getting a linker error with the above test, including if I seperate it out and try to build it seperately:
unresolved external symbol "char __cdel HYPRE_IJMatrixCreate(void)" ....
Ok, it looks like a problem with either the library or linker commands. But here's the interesting thing - If I transplant this code into VS, with the same project setting that allows it to build the much more nontrivial Hypre example programs, I get the same error:
Error LNK2001 unresolved external symbol "char __cdecl HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1 C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1
So it seems like there might be something about this type of stub program that is not working with my Hypre library. I don't fully understand this program - it's able to call the function with no arguments, but it also needs to be linked against a library containing the function, apparently by wrapping it in a static void function? Not something I've seen before.
Does anyone have any insight into what might be going wrong - or really just any explaination of how the stub program works so I can figure out why it isn't in this case?
Many thanks,
Daniel
Ok, so the test does exhibit different errors when run through the config script, compared to my attempts to isolate it: ---------------------------- Executing: [compilation command] successful compile: Source: #include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */ char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); } int main() { _check_HYPRE_IJMatrixCreate();; return 0; } Executing: [linker command] stdout: LINK : warning LNK4098: defaultlib 'msvcrtd.lib' conflicts with use of other libs; use /NODEFAULTLIB:library LINK : warning LNK4286: symbol '__stdio_common_vsprintf' defined in 'libucrt.lib(output.obj)' is imported by 'HYPRE.lib(dtrmm.c.obj)' LINK : warning LNK4286: symbol '__stdio_common_vsprintf' defined in 'libucrt.lib(output.obj)' is imported by 'HYPRE.lib(dtrmv.c.obj)' LINK : warning LNK4286: symbol '__stdio_common_vsprintf' defined in 'libucrt.lib(output.obj)' is imported by 'HYPRE.lib(lsame.c.obj)' LINK : warning LNK4286: symbol '__stdio_common_vsprintf' defined in 'libucrt.lib(output.obj)' is imported by 'HYPRE.lib(xerbla.c.obj)' LINK : warning LNK4286: symbol '__stdio_common_vsprintf' defined in 'libucrt.lib(output.obj)' is imported by 'HYPRE.lib(lsame.c.obj)' LINK : warning LNK4286: symbol '__stdio_common_vsprintf' defined in 'libucrt.lib(output.obj)' is imported by 'HYPRE.lib(dtrti2.c.obj)' [etc...] HYPRE.lib(par_vector.c.obj) : error LNK2019: unresolved external symbol __imp__wassert referenced in function hypre_ParVectorDestroy HYPRE.lib(vector.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(par_csr_matop.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(par_csr_communication.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(csr_matop.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(merge_sort.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(memory.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(IJMatrix_parcsr.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(par_csr_matrix.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(csr_matrix.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(memory.c.obj) : error LNK2019: unresolved external symbol __imp_realloc referenced in function hypre_ReAlloc HYPRE.lib(par_vector.c.obj) : error LNK2001: unresolved external symbol __imp_fopen HYPRE.lib(vector.c.obj) : error LNK2001: unresolved external symbol __imp_fopen HYPRE.lib(IJMatrix.c.obj) : error LNK2001: unresolved external symbol __imp_fopen HYPRE.lib(par_csr_matrix.c.obj) : error LNK2001: unresolved external symbol __imp_fopen HYPRE.lib(csr_matrix.c.obj) : error LNK2001: unresolved external symbol __imp_fopen HYPRE.lib(new_commpkg.c.obj) : error LNK2001: unresolved external symbol __imp_fopen HYPRE.lib(printf.c.obj) : error LNK2019: unresolved external symbol __imp___stdio_common_vfscanf referenced in function _vfscanf_l HYPRE.lib(printf.c.obj) : error LNK2019: unresolved external symbol __imp___stdio_common_vsscanf referenced in function _vsscanf_l HYPRE.lib(mmio.c.obj) : error LNK2001: unresolved external symbol __imp___stdio_common_vsscanf HYPRE.lib(mmio.c.obj) : error LNK2019: unresolved external symbol __imp_fgets referenced in function hypre_mm_read_banner HYPRE.lib(mmio.c.obj) : error LNK2019: unresolved external symbol __imp_tolower referenced in function hypre_mm_read_banner C:\cygwin64\tmp\PE0681~1\CONFIG~1.LIB\conftest.exe : fatal error LNK1120: 7 unresolved externals icx: error: linker command failed with exit code 1120 (use -v to see invocation) Possible ERROR while running linker: exit code 96 [more warnings etc] ------------------------- I've looked at the oneapi compiler documentation, and it seems like "nodefaultlib", "nostdlib", etc, options only work for linux versions of the compiler, so I'm not sure what to do about the first warning. From the errors, it looks like some core c or c++ library that hypre depends on isn't visible. I had some similar issues with ptscotch - but in that case I didn't have the warnings, and the errors gave me the names of libraries that were missing, which I lilnked in using the --cflags option (maybe --cc-linker-flags would have been neater, but it worked. I've tried both in order to try to get the above working). I can go into detail about the compile and linker commands if needed; I'd have to explain more about my choices for --cflags, etc too. I wonder if any of the above output shines any light on the hypre-is-shared-library hypothesis. Thanks, Daniel On Wed, Jul 19, 2023 at 4:58 PM Satish Balay via petsc-users < [email protected]> wrote:
I think it should work with static libraries and 64bit compilers.
That's how I think --download-f2cblaslapack [etc] work.
Also it works with MS-MPI - even-though its a dll install, the library stub provides this symbol somehow..
balay@ps5 /cygdrive/c/Program Files (x86)/Microsoft SDKs/MPI/Lib/x64 $ nm -Ao msmpi.lib |grep " MPI_Init" msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init_thread msmpi.lib:msmpi.dll:0000000000000000 T MPI_Initialized
However - if the library symbol is somehow mangled - this configure mode of checking library functions will fail.
Checking PETSc dll build:
balay@ps5 ~/petsc/arch-ci-mswin-uni/lib $ nm -Ao libpetsc.lib |grep MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 I __imp_MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 T MatCreateSeqAIJWithArrays
It also has the unmangled symbol - so I guess this mode can work generally with dlls.
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
Satish,
So it will always fail on Windows with Windows compilers (both with
static and shared libraries)? Is this true for all PETSc external packages? If so, why does the installation documentation say that some external packages can work with Windows compilers? (Presumably PETSc cannot since the configure tests will fail).
Barry
On Jul 19, 2023, at 11:40 AM, Satish Balay <[email protected]> wrote:
BTW: Some explanation of configure:
It attempts the following on linux:
Source: #include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */ char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } <<<<<<<
Note - it does not include 'HYPRE.h' here - but redefines the
prototype as 'char HYPRE_IJMatrixCreate();
Compiling it manually:
[balay@pj01 petsc]$ cat conftest.c char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } [balay@pj01 petsc]$ gcc -c conftest.c [balay@pj01 petsc]$ nm -Ao conftest.o |grep HYPRE_IJMatrixCreate conftest.o:0000000000000000 t _check_HYPRE_IJMatrixCreate conftest.o: U HYPRE_IJMatrixCreate [balay@pj01 petsc]$ nm -Ao arch-linux-c-debug/lib/libHYPRE.so |grep
HYPRE_IJMatrixCreate
arch-linux-c-debug/lib/libHYPRE.so:000000000007f2c2 T HYPRE_IJMatrixCreate [balay@pj01 petsc]$ <<<<
Here the "U HYPRE_IJMatrixCreate" in conftest.o matches "T HYPRE_IJMatrixCreate" in libHYPRE.so - so the "link" test in configure succeeds!
[balay@pj01 petsc]$ gcc -o conftest conftest.o arch-linux-c-debug/lib/libHYPRE.so [balay@pj01 petsc]$ echo $? 0 <<<<<
On windows - [due to name mangling by cdecl/stdcall, (/MT vs /MD) etc..] - this might not match - resulting in link failures.
Satish
On Wed, 19 Jul 2023, Satish Balay via petsc-users wrote:
You could try skipping this test [and assume --with-hypre-include and --with-hypre-lib options are correct] - and see if this works.
diff --git a/config/BuildSystem/config/packages/hypre.py b/config/BuildSystem/config/packages/hypre.py index 5bc88322aa2..2d6c7932e17 100644 --- a/config/BuildSystem/config/packages/hypre.py +++ b/config/BuildSystem/config/packages/hypre.py @@ -11,7 +11,7 @@ class Configure(config.package.GNUPackage): self.requiresversion = 1 self.gitcommit = 'v'+self.version self.download = ['git:// https://github.com/hypre-space/hypre',' https://github.com/hypre-space/hypre/archive/'+self.gitcommit+'.tar.gz'] - self.functions = ['HYPRE_IJMatrixCreate'] + self.functions = [] self.includes = ['HYPRE.h'] self.liblist = [['libHYPRE.a']] self.buildLanguages = ['C','Cxx']
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
You don't indicate what type of libraries you built hypre with;
static or shared. My guess is you ended up with shared
I think the answer to your difficulty is hidden in __cdecl (Satish
will know much better than me). When you are looking for symbols in Windows shared libraries you have to prepend something to the function prototype to have it successfully found. For example the PETSc include files have these things __declspec(dllimport) The configure test fails because it does not provide the needed prototype. Likely you built PTScotch with static libraries so no problem.
The simplest fix would be to build static hypre libraries. I think
it is a major project to get PETSc configure and macro system to work properly with external packages that are in Windows shared libraries since more use of __declspec would be needed.
Barry
The PETSc installation instructions should probably say something
about external packages with Windows shared libraries.
On Jul 19, 2023, at 10:52 AM, Daniel Stone <
[email protected]> wrote:
Hello,
I'm working on getting a petsc build running on windows. One
necessary package to include is Hypre. I've been able to build Hypre seperately using cmake, and confirmed that the library works
by setting up a VS project to run some of the example programs.
My attempted petsc build is being done through cygwin. I've been able to (with varying degrees of difficulty), build a fairly plain petsc, and one that downloads and builds ptscotch (after some modifications to both ptscotch and the config script). I am now attempting to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc). Note that the same compilers are being used for both Hypre and for petsc through cygwin - the new intel oneapi compilers (icx and ifx, after again varying amounts of pain to work around their awkwardness with the config script).
I'm seeing a problem when the config script does some tests on the included hypre lib. The source code looks like:
#include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */
#include "HYPRE.h"
char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main() { _check_HYPRE_IJMatrixCreate();; return 0; }
As I understand this is a fairly standard type of stub program used by the config script to check that it is able to link to certain symbols in given libraries. Tests like this have succeeded in my builds that include PTScotch.
I keep getting a linker error with the above test, including if I seperate it out and try to build it seperately:
unresolved external symbol "char __cdel HYPRE_IJMatrixCreate(void)" ....
Ok, it looks like a problem with either the library or linker commands. But here's the interesting thing - If I transplant this code into VS, with the same project setting that allows it to build the much more nontrivial Hypre example programs, I get the same error:
Error LNK2001 unresolved external symbol "char __cdecl HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1 C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1
So it seems like there might be something about this type of stub program that is not working with my Hypre library. I don't fully understand this program - it's able to call the function with no arguments, but it also needs to be linked against a library containing the function, apparently by wrapping it in a static void function? Not something I've seen before.
Does anyone have any insight into what might be going wrong - or really just any explaination of how the stub program works so I can figure out why it isn't in this case?
Many thanks,
Daniel
On Thu, 20 Jul 2023, Daniel Stone wrote:
Ok, so the test does exhibit different errors when run through the config script, compared to my attempts to isolate it:
----------------------------
Executing: [compilation command]
successful compile: Source: #include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */ char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main() { _check_HYPRE_IJMatrixCreate();; return 0; }
Executing: [linker command]
stdout: LINK : warning LNK4098: defaultlib 'msvcrtd.lib' conflicts with use of other libs; use /NODEFAULTLIB:library LINK : warning LNK4286: symbol '__stdio_common_vsprintf' defined in 'libucrt.lib(output.obj)' is imported by 'HYPRE.lib(dtrmm.c.obj)'
This error comes up when building different objfiles [or libraries] that go into a single binary with different compiler options - for ex: one with /MD - the other with /MT [each option sets and links with a different default compiler library - resulting in such conflicts] https://learn.microsoft.com/en-us/cpp/build/reference/md-mt-ld-use-run-time-... so the usual fix is to make sure all libraries, objfiles are compiled with the same compiler options Satish
LINK : warning LNK4286: symbol '__stdio_common_vsprintf' defined in 'libucrt.lib(output.obj)' is imported by 'HYPRE.lib(dtrmv.c.obj)' LINK : warning LNK4286: symbol '__stdio_common_vsprintf' defined in 'libucrt.lib(output.obj)' is imported by 'HYPRE.lib(lsame.c.obj)' LINK : warning LNK4286: symbol '__stdio_common_vsprintf' defined in 'libucrt.lib(output.obj)' is imported by 'HYPRE.lib(xerbla.c.obj)' LINK : warning LNK4286: symbol '__stdio_common_vsprintf' defined in 'libucrt.lib(output.obj)' is imported by 'HYPRE.lib(lsame.c.obj)' LINK : warning LNK4286: symbol '__stdio_common_vsprintf' defined in 'libucrt.lib(output.obj)' is imported by 'HYPRE.lib(dtrti2.c.obj)' [etc...] HYPRE.lib(par_vector.c.obj) : error LNK2019: unresolved external symbol __imp__wassert referenced in function hypre_ParVectorDestroy HYPRE.lib(vector.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(par_csr_matop.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(par_csr_communication.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(csr_matop.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(merge_sort.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(memory.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(IJMatrix_parcsr.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(par_csr_matrix.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(csr_matrix.c.obj) : error LNK2001: unresolved external symbol __imp__wassert HYPRE.lib(memory.c.obj) : error LNK2019: unresolved external symbol __imp_realloc referenced in function hypre_ReAlloc HYPRE.lib(par_vector.c.obj) : error LNK2001: unresolved external symbol __imp_fopen HYPRE.lib(vector.c.obj) : error LNK2001: unresolved external symbol __imp_fopen HYPRE.lib(IJMatrix.c.obj) : error LNK2001: unresolved external symbol __imp_fopen HYPRE.lib(par_csr_matrix.c.obj) : error LNK2001: unresolved external symbol __imp_fopen HYPRE.lib(csr_matrix.c.obj) : error LNK2001: unresolved external symbol __imp_fopen HYPRE.lib(new_commpkg.c.obj) : error LNK2001: unresolved external symbol __imp_fopen HYPRE.lib(printf.c.obj) : error LNK2019: unresolved external symbol __imp___stdio_common_vfscanf referenced in function _vfscanf_l HYPRE.lib(printf.c.obj) : error LNK2019: unresolved external symbol __imp___stdio_common_vsscanf referenced in function _vsscanf_l HYPRE.lib(mmio.c.obj) : error LNK2001: unresolved external symbol __imp___stdio_common_vsscanf HYPRE.lib(mmio.c.obj) : error LNK2019: unresolved external symbol __imp_fgets referenced in function hypre_mm_read_banner HYPRE.lib(mmio.c.obj) : error LNK2019: unresolved external symbol __imp_tolower referenced in function hypre_mm_read_banner C:\cygwin64\tmp\PE0681~1\CONFIG~1.LIB\conftest.exe : fatal error LNK1120: 7 unresolved externals icx: error: linker command failed with exit code 1120 (use -v to see invocation) Possible ERROR while running linker: exit code 96 [more warnings etc]
-------------------------
I've looked at the oneapi compiler documentation, and it seems like "nodefaultlib", "nostdlib", etc, options only work for linux versions of the compiler, so I'm not sure what to do about the first warning. From the errors, it looks like some core c or c++ library that hypre depends on isn't visible. I had some similar issues with ptscotch - but in that case I didn't have the warnings, and the errors gave me the names of libraries that were missing, which I lilnked in using the --cflags option (maybe --cc-linker-flags would have been neater, but it worked. I've tried both in order to try to get the above working).
I can go into detail about the compile and linker commands if needed; I'd have to explain more about my choices for --cflags, etc too. I wonder if any of the above output shines any light on the hypre-is-shared-library hypothesis.
Thanks,
Daniel
On Wed, Jul 19, 2023 at 4:58 PM Satish Balay via petsc-users < [email protected]> wrote:
I think it should work with static libraries and 64bit compilers.
That's how I think --download-f2cblaslapack [etc] work.
Also it works with MS-MPI - even-though its a dll install, the library stub provides this symbol somehow..
balay@ps5 /cygdrive/c/Program Files (x86)/Microsoft SDKs/MPI/Lib/x64 $ nm -Ao msmpi.lib |grep " MPI_Init" msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init msmpi.lib:msmpi.dll:0000000000000000 T MPI_Init_thread msmpi.lib:msmpi.dll:0000000000000000 T MPI_Initialized
However - if the library symbol is somehow mangled - this configure mode of checking library functions will fail.
Checking PETSc dll build:
balay@ps5 ~/petsc/arch-ci-mswin-uni/lib $ nm -Ao libpetsc.lib |grep MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 I __imp_MatCreateSeqAIJWithArrays libpetsc.lib:libpetsc.dll:0000000000000000 T MatCreateSeqAIJWithArrays
It also has the unmangled symbol - so I guess this mode can work generally with dlls.
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
Satish,
So it will always fail on Windows with Windows compilers (both with
static and shared libraries)? Is this true for all PETSc external packages? If so, why does the installation documentation say that some external packages can work with Windows compilers? (Presumably PETSc cannot since the configure tests will fail).
Barry
On Jul 19, 2023, at 11:40 AM, Satish Balay <[email protected]> wrote:
BTW: Some explanation of configure:
It attempts the following on linux:
Source: #include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */ char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } <<<<<<<
Note - it does not include 'HYPRE.h' here - but redefines the
prototype as 'char HYPRE_IJMatrixCreate();
Compiling it manually:
[balay@pj01 petsc]$ cat conftest.c char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main(void) { _check_HYPRE_IJMatrixCreate(); return 0; } [balay@pj01 petsc]$ gcc -c conftest.c [balay@pj01 petsc]$ nm -Ao conftest.o |grep HYPRE_IJMatrixCreate conftest.o:0000000000000000 t _check_HYPRE_IJMatrixCreate conftest.o: U HYPRE_IJMatrixCreate [balay@pj01 petsc]$ nm -Ao arch-linux-c-debug/lib/libHYPRE.so |grep
HYPRE_IJMatrixCreate
arch-linux-c-debug/lib/libHYPRE.so:000000000007f2c2 T HYPRE_IJMatrixCreate [balay@pj01 petsc]$ <<<<
Here the "U HYPRE_IJMatrixCreate" in conftest.o matches "T HYPRE_IJMatrixCreate" in libHYPRE.so - so the "link" test in configure succeeds!
[balay@pj01 petsc]$ gcc -o conftest conftest.o arch-linux-c-debug/lib/libHYPRE.so [balay@pj01 petsc]$ echo $? 0 <<<<<
On windows - [due to name mangling by cdecl/stdcall, (/MT vs /MD) etc..] - this might not match - resulting in link failures.
Satish
On Wed, 19 Jul 2023, Satish Balay via petsc-users wrote:
You could try skipping this test [and assume --with-hypre-include and --with-hypre-lib options are correct] - and see if this works.
diff --git a/config/BuildSystem/config/packages/hypre.py b/config/BuildSystem/config/packages/hypre.py index 5bc88322aa2..2d6c7932e17 100644 --- a/config/BuildSystem/config/packages/hypre.py +++ b/config/BuildSystem/config/packages/hypre.py @@ -11,7 +11,7 @@ class Configure(config.package.GNUPackage): self.requiresversion = 1 self.gitcommit = 'v'+self.version self.download = ['git:// https://github.com/hypre-space/hypre',' https://github.com/hypre-space/hypre/archive/'+self.gitcommit+'.tar.gz'] - self.functions = ['HYPRE_IJMatrixCreate'] + self.functions = [] self.includes = ['HYPRE.h'] self.liblist = [['libHYPRE.a']] self.buildLanguages = ['C','Cxx']
Satish
On Wed, 19 Jul 2023, Barry Smith wrote:
You don't indicate what type of libraries you built hypre with;
static or shared. My guess is you ended up with shared
I think the answer to your difficulty is hidden in __cdecl (Satish
will know much better than me). When you are looking for symbols in Windows shared libraries you have to prepend something to the function prototype to have it successfully found. For example the PETSc include files have these things __declspec(dllimport) The configure test fails because it does not provide the needed prototype. Likely you built PTScotch with static libraries so no problem.
The simplest fix would be to build static hypre libraries. I think
it is a major project to get PETSc configure and macro system to work properly with external packages that are in Windows shared libraries since more use of __declspec would be needed.
Barry
The PETSc installation instructions should probably say something
about external packages with Windows shared libraries.
On Jul 19, 2023, at 10:52 AM, Daniel Stone <
[email protected]> wrote:
Hello,
I'm working on getting a petsc build running on windows. One
necessary package to include is Hypre. I've been able to build Hypre seperately using cmake, and confirmed that the library works
by setting up a VS project to run some of the example programs.
My attempted petsc build is being done through cygwin. I've been able to (with varying degrees of difficulty), build a fairly plain petsc, and one that downloads and builds ptscotch (after some modifications to both ptscotch and the config script). I am now attempting to include Hypre (using the --hypre-iclude and --hypre-lib flags, etc). Note that the same compilers are being used for both Hypre and for petsc through cygwin - the new intel oneapi compilers (icx and ifx, after again varying amounts of pain to work around their awkwardness with the config script).
I'm seeing a problem when the config script does some tests on the included hypre lib. The source code looks like:
#include "confdefs.h" #include "conffix.h" /* Override any gcc2 internal prototype to avoid an error. */
#include "HYPRE.h"
char HYPRE_IJMatrixCreate(); static void _check_HYPRE_IJMatrixCreate() { HYPRE_IJMatrixCreate(); }
int main() { _check_HYPRE_IJMatrixCreate();; return 0; }
As I understand this is a fairly standard type of stub program used by the config script to check that it is able to link to certain symbols in given libraries. Tests like this have succeeded in my builds that include PTScotch.
I keep getting a linker error with the above test, including if I seperate it out and try to build it seperately:
unresolved external symbol "char __cdel HYPRE_IJMatrixCreate(void)" ....
Ok, it looks like a problem with either the library or linker commands. But here's the interesting thing - If I transplant this code into VS, with the same project setting that allows it to build the much more nontrivial Hypre example programs, I get the same error:
Error LNK2001 unresolved external symbol "char __cdecl HYPRE_IJMatrixCreate(void)" (?HYPRE_IJMatrixCreate@@YADXZ) hypretry1 C:\Users\DanielOGS\source\repos\hypretry1\hypretry1\Source.obj 1
So it seems like there might be something about this type of stub program that is not working with my Hypre library. I don't fully understand this program - it's able to call the function with no arguments, but it also needs to be linked against a library containing the function, apparently by wrapping it in a static void function? Not something I've seen before.
Does anyone have any insight into what might be going wrong - or really just any explaination of how the stub program works so I can figure out why it isn't in this case?
Many thanks,
Daniel
participants (3)
-
Barry Smith -
Daniel Stone -
Satish Balay