June 2017 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] How to configure MPICH build to create linux x86 cross-compiler for ARM?
by Kenneth Raffenetti 06 Jun '17

06 Jun '17

For Fortran, the 'maint/fcrosscompile' directory contains a configure script you can run on the target architecture to generate a cross file. Unfortunately it looks like there is a bug that needs fixing. I can put together a pull request soon to address the issue. Ken On 06/05/2017 10:13 AM, Alexander Rast wrote: > Yes, I knew that mpicc isn't building its own compiler from the ground > up. I reckoned though that maybe since the --target option in normal > builds came closest to expressing the result I'm trying to achieve, > maybe that's what you needed to do. I'd tried the --host option as well > before but that errored when I tried to run mpicc. > > What steps are needed to configure for Fortran? I tried setting several > CROSS_F77_xxx flags but then ended up with the uninformative error > during configure: > > configure: error: '' is empty. > > C cross-compilers would be adequate for early testing but the platform > we are building is going to amongst other things going to be compiling > and linking source files provided by external users, many of whom will > undoubtely be using Fortran so in the long run it's going to be > essential to have that cross-compiler available as well. > > > > On Fri, Jun 2, 2017 at 4:15 PM, Kenneth Raffenetti <raffenet(a)mcs.anl.gov > <mailto:[email protected]>> wrote: > > Hi Alex, > > One thing to understand is that MPICH does not actually build > compiler tools, so the --target option is not used. mpicc and > friends are only wrapper scripts. Here is how I was able to > configure/build MPICH on my laptop (Ubuntu 16.10) to cross-compile > binaries arm-linux-gnueabi. Fortran is disabled since it requires > additional steps. > > ./configure --build=x86_64-linux-gnu --host=arm-linux-gnueabi > CC=/usr/bin/arm-linux-gnueabi-gcc-6 > CXX=/usr/bin/arm-linux-gnueabi-g++-6 --disable-fortran > > Ken > > On 06/01/2017 10:22 AM, Alexander Rast wrote: > > I assume you mean from the build intended for the ARM targets. > In any case here they are. > > On Thu, Jun 1, 2017 at 4:04 PM, Kenneth Raffenetti > <raffenet(a)mcs.anl.gov <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>>> wrote: > > Hi Alex, > > I don't have experience with this type of build, but I can > try to > replicate it on a system here internally. Can you provide a > couple > of artifacts first? > > 1. the config.log file from the top-level build directory > 2. the output of 'mpicc -show' from the installed build > > Thanks, > Ken > > On 06/01/2017 10:00 AM, Alexander Rast wrote: > > All, > > I'm trying to build MPICH so that I can cross-compile MPI > applications on my x86_64-linux system to run on an > arm-linux > machine. I want to emphasise that this is different from > building a native ARM MPICH install that runs on ARM > itself; the > intent is that an x86-linux 'root' process compiles ARM MPI > application binaries and distributes them to the ARM > cores. I > have a separate build for compiling the MPI binaries > that lie on > the root x86 system. I tried running configure with > --build=x86_64-linux-gnu --host=x86_64-linux-gnu > --target=arm-linux-gnueabi, but that didn't give me > what I need. > If, meanwhile, I try to run mpicc off the native x86 > build (the > MPICH installation used for running MPI applications on > the root > x86 system) it complains that the target type specified > in the > makefile (CFLAGS = -march=armv8-a -mtune=cortex-a53) isn't > supported. It's not clear how one creates this sort of > 'hybrid' > build, running on one platform, compiling MPI > applications for > another. > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > <mailto:[email protected]> <mailto:[email protected] > <mailto:[email protected]>> > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > <https://lists.mpich.org/mailman/listinfo/discuss> > <https://lists.mpich.org/mailman/listinfo/discuss > <https://lists.mpich.org/mailman/listinfo/discuss>> > > > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] How to configure MPICH build to create linux x86 cross-compiler for ARM?
by Alexander Rast 05 Jun '17

05 Jun '17

Yes, I knew that mpicc isn't building its own compiler from the ground up. I reckoned though that maybe since the --target option in normal builds came closest to expressing the result I'm trying to achieve, maybe that's what you needed to do. I'd tried the --host option as well before but that errored when I tried to run mpicc. What steps are needed to configure for Fortran? I tried setting several CROSS_F77_xxx flags but then ended up with the uninformative error during configure: configure: error: '' is empty. C cross-compilers would be adequate for early testing but the platform we are building is going to amongst other things going to be compiling and linking source files provided by external users, many of whom will undoubtely be using Fortran so in the long run it's going to be essential to have that cross-compiler available as well. On Fri, Jun 2, 2017 at 4:15 PM, Kenneth Raffenetti <raffenet(a)mcs.anl.gov> wrote: > Hi Alex, > > One thing to understand is that MPICH does not actually build compiler > tools, so the --target option is not used. mpicc and friends are only > wrapper scripts. Here is how I was able to configure/build MPICH on my > laptop (Ubuntu 16.10) to cross-compile binaries arm-linux-gnueabi. Fortran > is disabled since it requires additional steps. > > ./configure --build=x86_64-linux-gnu --host=arm-linux-gnueabi > CC=/usr/bin/arm-linux-gnueabi-gcc-6 CXX=/usr/bin/arm-linux-gnueabi-g++-6 > --disable-fortran > > Ken > > On 06/01/2017 10:22 AM, Alexander Rast wrote: > >> I assume you mean from the build intended for the ARM targets. In any >> case here they are. >> >> On Thu, Jun 1, 2017 at 4:04 PM, Kenneth Raffenetti <raffenet(a)mcs.anl.gov >> <mailto:[email protected]>> wrote: >> >> Hi Alex, >> >> I don't have experience with this type of build, but I can try to >> replicate it on a system here internally. Can you provide a couple >> of artifacts first? >> >> 1. the config.log file from the top-level build directory >> 2. the output of 'mpicc -show' from the installed build >> >> Thanks, >> Ken >> >> On 06/01/2017 10:00 AM, Alexander Rast wrote: >> >> All, >> >> I'm trying to build MPICH so that I can cross-compile MPI >> applications on my x86_64-linux system to run on an arm-linux >> machine. I want to emphasise that this is different from >> building a native ARM MPICH install that runs on ARM itself; the >> intent is that an x86-linux 'root' process compiles ARM MPI >> application binaries and distributes them to the ARM cores. I >> have a separate build for compiling the MPI binaries that lie on >> the root x86 system. I tried running configure with >> --build=x86_64-linux-gnu --host=x86_64-linux-gnu >> --target=arm-linux-gnueabi, but that didn't give me what I need. >> If, meanwhile, I try to run mpicc off the native x86 build (the >> MPICH installation used for running MPI applications on the root >> x86 system) it complains that the target type specified in the >> makefile (CFLAGS = -march=armv8-a -mtune=cortex-a53) isn't >> supported. It's not clear how one creates this sort of 'hybrid' >> build, running on one platform, compiling MPI applications for >> another. >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> <https://lists.mpich.org/mailman/listinfo/discuss> >> >> >>

1 0

Re: [mpich-discuss] How to configure MPICH build to create linux x86 cross-compiler for ARM?
by Kenneth Raffenetti 02 Jun '17

02 Jun '17

After installing the 'crossbuild-essential-armel' package, configure finds the right compilers without the need to specify them, so this should also work: ./configure --build=x86_64-linux-gnu --host=arm-linux-gnueabi --disable-fortran file examples/cpi examples/cpi: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.3, for GNU/Linux 3.2.0, BuildID[sha1]=b24df425c14a6754bc97dce22580827d3d65dbfb, not stripped mpicc -show arm-linux-gnueabi-gcc -I/sandbox/mpich/i/include -L/sandbox/mpich/i/lib -Wl,-rpath -Wl,/sandbox/mpich/i/lib -Wl,--enable-new-dtags -lmpi Ken On 06/02/2017 10:15 AM, Kenneth Raffenetti wrote: > Hi Alex, > > One thing to understand is that MPICH does not actually build compiler > tools, so the --target option is not used. mpicc and friends are only > wrapper scripts. Here is how I was able to configure/build MPICH on my > laptop (Ubuntu 16.10) to cross-compile binaries arm-linux-gnueabi. > Fortran is disabled since it requires additional steps. > > ./configure --build=x86_64-linux-gnu --host=arm-linux-gnueabi > CC=/usr/bin/arm-linux-gnueabi-gcc-6 CXX=/usr/bin/arm-linux-gnueabi-g++-6 > --disable-fortran > > Ken > > On 06/01/2017 10:22 AM, Alexander Rast wrote: >> I assume you mean from the build intended for the ARM targets. In any >> case here they are. >> >> On Thu, Jun 1, 2017 at 4:04 PM, Kenneth Raffenetti >> <raffenet(a)mcs.anl.gov <mailto:[email protected]>> wrote: >> >> Hi Alex, >> >> I don't have experience with this type of build, but I can try to >> replicate it on a system here internally. Can you provide a couple >> of artifacts first? >> >> 1. the config.log file from the top-level build directory >> 2. the output of 'mpicc -show' from the installed build >> >> Thanks, >> Ken >> >> On 06/01/2017 10:00 AM, Alexander Rast wrote: >> >> All, >> >> I'm trying to build MPICH so that I can cross-compile MPI >> applications on my x86_64-linux system to run on an arm-linux >> machine. I want to emphasise that this is different from >> building a native ARM MPICH install that runs on ARM itself; the >> intent is that an x86-linux 'root' process compiles ARM MPI >> application binaries and distributes them to the ARM cores. I >> have a separate build for compiling the MPI binaries that lie on >> the root x86 system. I tried running configure with >> --build=x86_64-linux-gnu --host=x86_64-linux-gnu >> --target=arm-linux-gnueabi, but that didn't give me what I need. >> If, meanwhile, I try to run mpicc off the native x86 build (the >> MPICH installation used for running MPI applications on the root >> x86 system) it complains that the target type specified in the >> makefile (CFLAGS = -march=armv8-a -mtune=cortex-a53) isn't >> supported. It's not clear how one creates this sort of 'hybrid' >> build, running on one platform, compiling MPI applications for >> another. >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> <https://lists.mpich.org/mailman/listinfo/discuss> >> >> > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] How to configure MPICH build to create linux x86 cross-compiler for ARM?
by Kenneth Raffenetti 02 Jun '17

02 Jun '17

Hi Alex, One thing to understand is that MPICH does not actually build compiler tools, so the --target option is not used. mpicc and friends are only wrapper scripts. Here is how I was able to configure/build MPICH on my laptop (Ubuntu 16.10) to cross-compile binaries arm-linux-gnueabi. Fortran is disabled since it requires additional steps. ./configure --build=x86_64-linux-gnu --host=arm-linux-gnueabi CC=/usr/bin/arm-linux-gnueabi-gcc-6 CXX=/usr/bin/arm-linux-gnueabi-g++-6 --disable-fortran Ken On 06/01/2017 10:22 AM, Alexander Rast wrote: > I assume you mean from the build intended for the ARM targets. In any > case here they are. > > On Thu, Jun 1, 2017 at 4:04 PM, Kenneth Raffenetti <raffenet(a)mcs.anl.gov > <mailto:[email protected]>> wrote: > > Hi Alex, > > I don't have experience with this type of build, but I can try to > replicate it on a system here internally. Can you provide a couple > of artifacts first? > > 1. the config.log file from the top-level build directory > 2. the output of 'mpicc -show' from the installed build > > Thanks, > Ken > > On 06/01/2017 10:00 AM, Alexander Rast wrote: > > All, > > I'm trying to build MPICH so that I can cross-compile MPI > applications on my x86_64-linux system to run on an arm-linux > machine. I want to emphasise that this is different from > building a native ARM MPICH install that runs on ARM itself; the > intent is that an x86-linux 'root' process compiles ARM MPI > application binaries and distributes them to the ARM cores. I > have a separate build for compiling the MPI binaries that lie on > the root x86 system. I tried running configure with > --build=x86_64-linux-gnu --host=x86_64-linux-gnu > --target=arm-linux-gnueabi, but that didn't give me what I need. > If, meanwhile, I try to run mpicc off the native x86 build (the > MPICH installation used for running MPI applications on the root > x86 system) it complains that the target type specified in the > makefile (CFLAGS = -march=armv8-a -mtune=cortex-a53) isn't > supported. It's not clear how one creates this sort of 'hybrid' > build, running on one platform, compiling MPI applications for > another. > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org <mailto:[email protected]> > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > <https://lists.mpich.org/mailman/listinfo/discuss> > > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Fwd: MPICH fault tolerance and resiliency
by sanjeev s 01 Jun '17

01 Jun '17

Thanks for the info. Regards Sanjeev On Thu, Jun 1, 2017 at 9:52 PM, Guo, Yanfei <yguo(a)anl.gov> wrote: > There is no support for fault tolerance in the current MPI standard. But > there will be one in the future. > > Yanfei Guo > Postdoctoral Researcher > MCS Division, ANL > > > On 6/1/17, 1:28 AM, "sanjeev s" <snjv.workmail(a)gmail.com> wrote: > > Thanks for the reply and valuable suggestion. > > > One more Question: > > > Anyway to avoid bringing down(cleaning up ) all MPI jobs when one job dies. > > > > > Regards > Sanjeev > > > > On Thu, Jun 1, 2017 at 10:25 AM, Guo, Yanfei > <yguo(a)anl.gov> wrote: > > On 5/30/17, 10:19 PM, "sanjeev s" <snjv.workmail(a)gmail.com> wrote: > > > Hi, > > > > I have added the detailed use case.. > > > > > I did not follow what were you trying to do here. Can you be more > specific? > > > Suppose I have an application which accepts some request from client and > do > > the MPI job on request arrival(lazy load of mpi lib). Now I started this > > application through mpiexec on 4 machines. > > mpiexec will start my application on 4 machines not MPI jobs. On request > > arrival I will distribute MPI jobs to this cluster after doing some > processing > > in my application on master node.(1 of out of 4 I will designate as > master > > node). Now I want to add one more machine( having my application) to > this > > cluster, without stopping my already running applications. My master > process > > should be able to see this 5th machine and distribute the task. Is there > any > > way to achieve this? > > Is this "application" an MPI program? If so, there are two options. 1. > Starting > the MPI process on the 5th machine and have it connect to your existing > processes (with MPI_Comm_connect/MPI_Comm_accept). 2. Letting the existing > processes spawn new processes on the new machine (with MPI_Comm_spawn). > You can > check out the book "Using Advanced MPI" for details and examples. If the > aforementioned application is not an MPI program, then what you are trying > to > do is beyond the scope of MPI. > > > > > > Also, When I took the size(number of instance for that comm), I am not > > > getting the count for client instance. To distribute the task, I need > to have > > > additional logic to handle this case in my application. > > > Can you be more specific about which comm you were referring to? > > > > > I am referring to MPI_COMM_WORLD. Is there any resize kind of api which > will > > give me how many jobs are there on this comm after doing one accept > connection? > > MPI_COMM_WORLD is created during MPI_Init. It does not change when a new > process group is connected. The communicator that accepts the connection > will > give you an intercommunicator which allows the accepting processes to talk > to > the connecting processes. You can merge this intercommunicator to a > intracommunicator that contains all the processes. > > > Hope this helps in understanding the problem. > > > > Regards > > Sanjeev > > Yanfei Guo > > > > > On Mon, May 29, 2017 at 8:48 AM, sanjeev s > <snjv.workmail(a)gmail.com> wrote: > > Hi, > > > Thanks for the reply. I have given the detailed use case below. > > > Regards > Sanjeev > > > > > > On Sat, May 27, 2017 at 2:30 AM, Guo, Yanfei > <yguo(a)anl.gov> wrote: > > Hi Sanjeev, > > Please see the inline comments below. > > Yanfei Guo > Postdoctoral Researcher > MCS Division, ANL > > > On 5/26/17, 10:40 AM, "sanjeev s" <snjv.workmail(a)gmail.com> wrote: > > > Hi, > > > > In dynamic process, I read about two models: Client server and parent > child. > > > > Client Server : We need to have dedicated threads each for client and > server. Now considering all instances same , we will end up doing lot of > thread creation apart from our application worker threads. > > I assume you are referring to the connect/accept model by client/server. > In that case, you may need one thread on the server side to handle the > incoming connection because the `MPI_Comm_accept` call > is blocking. A dedicated thread is not necessary in the client. > It is your application’s job to decide who is the server. > > > > > > Yes I am referring to same connect/accept model. I don't want to put this > logic in our application. I want some external process manager ( like > Hydra) which can handle this form me .Please see below for detail use case. > > > > > > Moreover when 1 instance (app) goes down, we want > that instance to come up without doing much manual work. We don't want to > club this logic in our application. > > I did not follow what were you trying to do here. > Can you be more specific? > > > > Suppose I have an application which accepts some request from client and > do the MPI job on request arrival(lazy load of mpi lib). Now I started this > application through mpiexec on 4 machines. > mpiexec will start my application on 4 machines not MPI jobs. On request > arrival I will distribute MPI jobs to this cluster after doing some > processing in my application on master node.(1 of out of 4 I will designate > as master node). Now I > want to add one more machine( having my application) to this cluster, > without stopping my already running applications. My master process should > be able to see this 5th machine and distribute the task. Is there any way > to achieve this? > > > > > > > Also, When I took the size(number of instance for that comm), I am not > getting the count for client instance. To distribute the task, I need to > have additional logic to handle this case in my application. > > Can you be more specific about which comm you were referring to? > > > > I am referring to MPI_COMM_WORLD. Is there any resize kind of api which > will give me how many jobs are there on this comm after doing one accept > connection? > > > > 2) Parent child: Suppose we have started 4 instance on 4 different > machines. Now we need to add another server. I don't think parent child/ > client server is good option in this case. > > > > We don't want to build process management capabilities in our > application. We are looking for process management in MPI itself (e.g in > Hydra )so that we can leverage on that. > > > > Please correct me if I am missing something in my understanding of > Dynamic model. > > > Regards > Sanjeev Sinha > > > > > > On Fri, May 26, 2017 at 8:46 PM, Halim Amer > <aamer(a)anl.gov> wrote: > > Sanjeev, > > > More precisely my requirement is suppose I started 4 instances of my > > application. Now I want to add one more instance dynamically to this set > > From my understanding, dynamic processes would work fine for this case. > Could you elaborate on why the dynamic process model is not sufficient for > your needs? > > Halim > > > www.mcs.anl.gov/~aamer <http://www.mcs.anl.gov/~aamer> < > http://www.mcs.anl.gov/~aamer> <http://www.mcs.anl.gov/~aamer> > > On 5/26/17 9:11 AM, sanjeev s wrote: > > > Hi mpich, > > I have a requirement where in we need to add start stop application > instances on the fly before starting a job.Is there any mpich service > available. I looked through dynamic process model, but its not sufficing > our need. > > More precisely my requirement is suppose I started 4 instances of my > application. Now I want to add one more instance dynamically to this set > > Is there any tool which MPICH supports for fault tolerance behavior? > > Thanks > Sanjeev > > > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > > > > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > > > > > > > > > > > > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > > > > > > > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss >

1 0

Re: [mpich-discuss] Fwd: MPICH fault tolerance and resiliency
by Guo, Yanfei 01 Jun '17

01 Jun '17

There is no support for fault tolerance in the current MPI standard. But there will be one in the future. Yanfei Guo Postdoctoral Researcher MCS Division, ANL On 6/1/17, 1:28 AM, "sanjeev s" <snjv.workmail(a)gmail.com> wrote: Thanks for the reply and valuable suggestion. One more Question: Anyway to avoid bringing down(cleaning up ) all MPI jobs when one job dies. Regards Sanjeev On Thu, Jun 1, 2017 at 10:25 AM, Guo, Yanfei <yguo(a)anl.gov> wrote: On 5/30/17, 10:19 PM, "sanjeev s" <snjv.workmail(a)gmail.com> wrote: > Hi, > I have added the detailed use case.. > > I did not follow what were you trying to do here. Can you be more specific? > Suppose I have an application which accepts some request from client and do > the MPI job on request arrival(lazy load of mpi lib). Now I started this > application through mpiexec on 4 machines. > mpiexec will start my application on 4 machines not MPI jobs. On request > arrival I will distribute MPI jobs to this cluster after doing some processing > in my application on master node.(1 of out of 4 I will designate as master > node). Now I want to add one more machine( having my application) to this > cluster, without stopping my already running applications. My master process > should be able to see this 5th machine and distribute the task. Is there any > way to achieve this? Is this "application" an MPI program? If so, there are two options. 1. Starting the MPI process on the 5th machine and have it connect to your existing processes (with MPI_Comm_connect/MPI_Comm_accept). 2. Letting the existing processes spawn new processes on the new machine (with MPI_Comm_spawn). You can check out the book "Using Advanced MPI" for details and examples. If the aforementioned application is not an MPI program, then what you are trying to do is beyond the scope of MPI. > > Also, When I took the size(number of instance for that comm), I am not > > getting the count for client instance. To distribute the task, I need to have > > additional logic to handle this case in my application. > Can you be more specific about which comm you were referring to? > I am referring to MPI_COMM_WORLD. Is there any resize kind of api which will > give me how many jobs are there on this comm after doing one accept connection? MPI_COMM_WORLD is created during MPI_Init. It does not change when a new process group is connected. The communicator that accepts the connection will give you an intercommunicator which allows the accepting processes to talk to the connecting processes. You can merge this intercommunicator to a intracommunicator that contains all the processes. > Hope this helps in understanding the problem. > Regards > Sanjeev Yanfei Guo On Mon, May 29, 2017 at 8:48 AM, sanjeev s <snjv.workmail(a)gmail.com> wrote: Hi, Thanks for the reply. I have given the detailed use case below. Regards Sanjeev On Sat, May 27, 2017 at 2:30 AM, Guo, Yanfei <yguo(a)anl.gov> wrote: Hi Sanjeev, Please see the inline comments below. Yanfei Guo Postdoctoral Researcher MCS Division, ANL On 5/26/17, 10:40 AM, "sanjeev s" <snjv.workmail(a)gmail.com> wrote: > Hi, > In dynamic process, I read about two models: Client server and parent child. > Client Server : We need to have dedicated threads each for client and server. Now considering all instances same , we will end up doing lot of thread creation apart from our application worker threads. I assume you are referring to the connect/accept model by client/server. In that case, you may need one thread on the server side to handle the incoming connection because the `MPI_Comm_accept` call is blocking. A dedicated thread is not necessary in the client. It is your application’s job to decide who is the server. Yes I am referring to same connect/accept model. I don't want to put this logic in our application. I want some external process manager ( like Hydra) which can handle this form me .Please see below for detail use case. > Moreover when 1 instance (app) goes down, we want that instance to come up without doing much manual work. We don't want to club this logic in our application. I did not follow what were you trying to do here. Can you be more specific? Suppose I have an application which accepts some request from client and do the MPI job on request arrival(lazy load of mpi lib). Now I started this application through mpiexec on 4 machines. mpiexec will start my application on 4 machines not MPI jobs. On request arrival I will distribute MPI jobs to this cluster after doing some processing in my application on master node.(1 of out of 4 I will designate as master node). Now I want to add one more machine( having my application) to this cluster, without stopping my already running applications. My master process should be able to see this 5th machine and distribute the task. Is there any way to achieve this? > Also, When I took the size(number of instance for that comm), I am not getting the count for client instance. To distribute the task, I need to have additional logic to handle this case in my application. Can you be more specific about which comm you were referring to? I am referring to MPI_COMM_WORLD. Is there any resize kind of api which will give me how many jobs are there on this comm after doing one accept connection? > 2) Parent child: Suppose we have started 4 instance on 4 different machines. Now we need to add another server. I don't think parent child/ client server is good option in this case. > We don't want to build process management capabilities in our application. We are looking for process management in MPI itself (e.g in Hydra )so that we can leverage on that. > Please correct me if I am missing something in my understanding of Dynamic model. Regards Sanjeev Sinha On Fri, May 26, 2017 at 8:46 PM, Halim Amer <aamer(a)anl.gov> wrote: Sanjeev, > More precisely my requirement is suppose I started 4 instances of my > application. Now I want to add one more instance dynamically to this set From my understanding, dynamic processes would work fine for this case. Could you elaborate on why the dynamic process model is not sufficient for your needs? Halim www.mcs.anl.gov/~aamer <http://www.mcs.anl.gov/~aamer> <http://www.mcs.anl.gov/~aamer> <http://www.mcs.anl.gov/~aamer> On 5/26/17 9:11 AM, sanjeev s wrote: Hi mpich, I have a requirement where in we need to add start stop application instances on the fly before starting a job.Is there any mpich service available. I looked through dynamic process model, but its not sufficing our need. More precisely my requirement is suppose I started 4 instances of my application. Now I want to add one more instance dynamically to this set Is there any tool which MPICH supports for fault tolerance behavior? Thanks Sanjeev _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] How to configure MPICH build to create linux x86 cross-compiler for ARM?
by Alexander Rast 01 Jun '17

01 Jun '17

I assume you mean from the build intended for the ARM targets. In any case here they are. On Thu, Jun 1, 2017 at 4:04 PM, Kenneth Raffenetti <raffenet(a)mcs.anl.gov> wrote: > Hi Alex, > > I don't have experience with this type of build, but I can try to > replicate it on a system here internally. Can you provide a couple of > artifacts first? > > 1. the config.log file from the top-level build directory > 2. the output of 'mpicc -show' from the installed build > > Thanks, > Ken > > On 06/01/2017 10:00 AM, Alexander Rast wrote: > >> All, >> >> I'm trying to build MPICH so that I can cross-compile MPI applications on >> my x86_64-linux system to run on an arm-linux machine. I want to emphasise >> that this is different from building a native ARM MPICH install that runs >> on ARM itself; the intent is that an x86-linux 'root' process compiles ARM >> MPI application binaries and distributes them to the ARM cores. I have a >> separate build for compiling the MPI binaries that lie on the root x86 >> system. I tried running configure with --build=x86_64-linux-gnu >> --host=x86_64-linux-gnu --target=arm-linux-gnueabi, but that didn't give me >> what I need. If, meanwhile, I try to run mpicc off the native x86 build >> (the MPICH installation used for running MPI applications on the root x86 >> system) it complains that the target type specified in the makefile (CFLAGS >> = -march=armv8-a -mtune=cortex-a53) isn't supported. It's not clear how one >> creates this sort of 'hybrid' build, running on one platform, compiling MPI >> applications for another. >> >> >> _______________________________________________ >> discuss mailing list discuss(a)mpich.org >> To manage subscription options or unsubscribe: >> https://lists.mpich.org/mailman/listinfo/discuss >> >>

1 0

Re: [mpich-discuss] Unable to subscribe to mailing list
by Alex Rast 01 Jun '17

01 Jun '17

More information. I just sent an e-mail from alex.rast.technical(a)gmail.com. Now, it definitely reached the list, because *this* e-mail (adr1r17(a)soton.ac.uk) got the message a few seconds later. But neither an acknowledgement or post or any other message has come back to alex.rast.technical(a)gmail.com. Nor did anything appear in spam. Somehow any replies are disappearing into the aether. On 01/06/17 15:18, Thakur, Rajeev wrote: > You can have it send you your password from here: https://lists.mpich.org/mailman/options/discuss > > >> On Jun 1, 2017, at 9:14 AM, Alex Rast <adr1r17(a)soton.ac.uk> wrote: >> >> By the way, what should be done to log in to change e.g. account settings? I don't see, for example, a password. >> >> On 01/06/17 15:08, Thakur, Rajeev wrote: >>> I have added your address to the list manually. >>> Rajeev >>>> On Jun 1, 2017, at 8:58 AM, Alex Rast <adr1r17(a)soton.ac.uk> wrote: >>>> >>>> Yes. And no, nothing showed up. It's literally like the subscriptions are being sent into the void. I tried blind posting a message to the list from that account and also got absolutely no notification either way, neither a message bounce/refusal nor an acknowledgement. >>>> >>>> I'd have thought that if the system had objected in some way you'd get either an immediate message on the Web page when you pressed subscribe, or a 'sorry, your subscription was refused' or some other such message back to the proposed address, in any case, so that doesn't appear superficially to be the likely reason. >>>> >>>> On 31/05/17 23:25, Jeff Hammond wrote: >>>>> Have you checked your spam folder for the subscription confirmation email? >>>>> I am subscribed to this list using a gmail handle with a similar construction, so I doubt the system objects to your handle or host, although I've probably been subscribed for a decade at this point. >>>>> Jeff >>>>> On Wed, May 31, 2017 at 9:01 AM, Alex Rast <adr1r17(a)soton.ac.uk <mailto:[email protected]>> wrote: >>>>> Dear sirs, >>>>> I've only subscribed under this e-mail address in absolute >>>>> desperation, because the one I actually need to use: >>>>> alex.rast.technical(a)gmail.com >>>>> <mailto:[email protected]>, doesn't seem to work with >>>>> the system. I've tried subscribing multiple times with this approach >>>>> only to receive absolute silence and no indication either that >>>>> something went wrong. >>>>> What needs to be done to get the above-referenced e-mail subscribed >>>>> to the list? >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org <mailto:[email protected]> >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>>> <https://lists.mpich.org/mailman/listinfo/discuss> >>>>> -- >>>>> Jeff Hammond >>>>> jeff.science(a)gmail.com <mailto:[email protected]> >>>>> http://jeffhammond.github.io/ >>>>> _______________________________________________ >>>>> discuss mailing list discuss(a)mpich.org >>>>> To manage subscription options or unsubscribe: >>>>> https://lists.mpich.org/mailman/listinfo/discuss >>>> _______________________________________________ >>>> discuss mailing list discuss(a)mpich.org >>>> To manage subscription options or unsubscribe: >>>> https://lists.mpich.org/mailman/listinfo/discuss >>> _______________________________________________ >>> discuss mailing list discuss(a)mpich.org >>> To manage subscription options or unsubscribe: >>> https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] How to configure MPICH build to create linux x86 cross-compiler for ARM?
by Kenneth Raffenetti 01 Jun '17

01 Jun '17

Hi Alex, I don't have experience with this type of build, but I can try to replicate it on a system here internally. Can you provide a couple of artifacts first? 1. the config.log file from the top-level build directory 2. the output of 'mpicc -show' from the installed build Thanks, Ken On 06/01/2017 10:00 AM, Alexander Rast wrote: > All, > > I'm trying to build MPICH so that I can cross-compile MPI applications > on my x86_64-linux system to run on an arm-linux machine. I want to > emphasise that this is different from building a native ARM MPICH > install that runs on ARM itself; the intent is that an x86-linux 'root' > process compiles ARM MPI application binaries and distributes them to > the ARM cores. I have a separate build for compiling the MPI binaries > that lie on the root x86 system. I tried running configure with > --build=x86_64-linux-gnu --host=x86_64-linux-gnu > --target=arm-linux-gnueabi, but that didn't give me what I need. If, > meanwhile, I try to run mpicc off the native x86 build (the MPICH > installation used for running MPI applications on the root x86 system) > it complains that the target type specified in the makefile (CFLAGS = > -march=armv8-a -mtune=cortex-a53) isn't supported. It's not clear how > one creates this sort of 'hybrid' build, running on one platform, > compiling MPI applications for another. > > > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss > _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

How to configure MPICH build to create linux x86 cross-compiler for ARM?
by Alexander Rast 01 Jun '17

01 Jun '17

All, I'm trying to build MPICH so that I can cross-compile MPI applications on my x86_64-linux system to run on an arm-linux machine. I want to emphasise that this is different from building a native ARM MPICH install that runs on ARM itself; the intent is that an x86-linux 'root' process compiles ARM MPI application binaries and distributes them to the ARM cores. I have a separate build for compiling the MPI binaries that lie on the root x86 system. I tried running configure with --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=arm-linux-gnueabi, but that didn't give me what I need. If, meanwhile, I try to run mpicc off the native x86 build (the MPICH installation used for running MPI applications on the root x86 system) it complains that the target type specified in the makefile (CFLAGS = -march=armv8-a -mtune=cortex-a53) isn't supported. It's not clear how one creates this sort of 'hybrid' build, running on one platform, compiling MPI applications for another. _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0