November 2014 - discuss - mailman3.cels.anl.gov

Re: [mpich-discuss] Assertion failed in file src/mpid/ch3/src/ch3u_handle_send_req.c at line 61 (RMA && Derived datatypes)&In-Reply-To=<0A407957589BAB4F924824150C4293EF588072EC@CITESMBX2.ad.uillinois.edu>
by Balaji, Pavan 11 Nov '14

11 Nov '14

> On Nov 11, 2014, at 10:31 AM, Victor Vysotskiy <victor.vysotskiy(a)teokem.lu.se> wrote: > %mpirun -np 2 ./a.out > [1415719636.245549] [n1:37817:0] shm.c:65 MXM WARN Could not open the KNEM device file at /dev/knem : No such file or directory. Won't use knem. > > [1415719636.245549] [n1:37818:0] shm.c:65 MXM WARN Could not open the KNEM device file at /dev/knem : No such file or directory. Won't use knem. You can get rid of this by setting the environment variable MXM_LOG_LEVEL=error > [1415719636.263914] [n1:37817:0] ib_dev.c:443 MXM ERROR ibv_query_device() returned 38: No such file or directory > > [1415719636.264134] [n1:37818:0] ib_dev.c:443 MXM ERROR ibv_query_device() returned 38: No such file or directory These are weird. I’m assuming you have a Mellanox IB network? — Pavan -- Pavan Balaji ✉️ http://www.mcs.anl.gov/~balaji _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Assertion failed in file src/mpid/ch3/src/ch3u_handle_send_req.c at line 61 (RMA && Derived datatypes)&In-Reply-To=<0A407957589BAB4F924824150C4293EF588072EC@CITESMBX2.ad.uillinois.edu>
by Victor Vysotskiy 11 Nov '14

11 Nov '14

Dear Pavan, thank you for hints! Ok, I was able to install MXM locally without root privileges. Then I have configured and compiled MPICH3 with MXM. Everything went smoothly and finally I got a working binaries and libs. Unfortunately, root is still needed because a simple MPI HELLO WORLD code now crashes with the following error message: %mpirun -np 2 ./a.out [1415719636.245549] [n1:37817:0] shm.c:65 MXM WARN Could not open the KNEM device file at /dev/knem : No such file or directory. Won't use knem. [1415719636.245549] [n1:37818:0] shm.c:65 MXM WARN Could not open the KNEM device file at /dev/knem : No such file or directory. Won't use knem. [1415719636.263914] [n1:37817:0] ib_dev.c:443 MXM ERROR ibv_query_device() returned 38: No such file or directory [1415719636.264134] [n1:37818:0] ib_dev.c:443 MXM ERROR ibv_query_device() returned 38: No such file or directory Fatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(498).........: MPID_Init(187)................: channel initialization failed MPIDI_CH3_Init(89)............: MPID_nem_init(320)............: MPID_nem_mxm_init(157)........: MPID_nem_mxm_vc_terminate(451): mxm_init failed (Input/output error) Fatal error in MPI_Init: Other MPI error, error stack: MPIR_Init_thread(498).........: MPID_Init(187)................: channel initialization failed MPIDI_CH3_Init(89)............: MPID_nem_init(320)............: MPID_nem_mxm_init(157)........: MPID_nem_mxm_vc_terminate(451): mxm_init failed (Input/output error) %ldd a.out linux-vdso.so.1 => (0x00007ffff29ff000) libmpi.so.12 => /nobackup/global/x_vicvy/mpich3-dev-bin.mxm/lib/libmpi.so.12 (0x00007f9fd4fa1000) libm.so.6 => /lib64/libm.so.6 (0x00007f9fd4cfe000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f9fd4ae8000) libc.so.6 => /lib64/libc.so.6 (0x00007f9fd4754000) libdl.so.2 => /lib64/libdl.so.2 (0x00007f9fd454f000) libmxm.so.2 => /nobackup/global/x_vicvy/hpcx-v1.2.0-255-icc-MLNX_OFED_LINUX-2.3-1.5.0-redhat6.5/mxm/lib/libmxm.so.2 (0x00007f9fd41f5000) libz.so.1 => /lib64/libz.so.1 (0x00007f9fd3fdf000) libibverbs.so.1 => /usr/lib64/libibverbs.so.1 (0x00007f9fd3dd1000) librt.so.1 => /lib64/librt.so.1 (0x00007f9fd3bc9000) libgpfs.so => /usr/lib64/libgpfs.so (0x00007f9fd39ba000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f9fd379c000) libifport.so.5 => /software/apps/intel/composer_xe_2015.0.090/compiler/lib/intel64/libifport.so.5 (0x00007f9fd356f000) libifcore.so.5 => /software/apps/intel/composer_xe_2015.0.090/compiler/lib/intel64/libifcore.so.5 (0x00007f9fd3239000) libimf.so => /software/apps/intel/composer_xe_2015.0.090/compiler/lib/intel64/libimf.so (0x00007f9fd2d7e000) libsvml.so => /software/apps/intel/composer_xe_2015.0.090/compiler/lib/intel64/libsvml.so (0x00007f9fd212f000) libintlc.so.5 => /software/apps/intel/composer_xe_2015.0.090/compiler/lib/intel64/libintlc.so.5 (0x00007f9fd1ed5000) /lib64/ld-linux-x86-64.so.2 (0x00007f9fd5499000) libirng.so => /software/apps/intel/composer_xe_2015.0.090/compiler/lib/intel64/libirng.so (0x00007f9fd1ccd000) Since ’nemesis:ch3:ib’ is fixed, I will check it out. With best regards, Victor. _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] (no subject)
by Balaji, Pavan 11 Nov '14

11 Nov '14

Please only use this list for MPICH-related questions, not general programming questions. — Pavan > On Nov 11, 2014, at 9:42 AM, Chafik sanaa <san.chafik(a)gmail.com> wrote: > > Hi, > What should I do? i have the folowing error (error C4703: variable de pointeur locale potentiellement non initialisée 'sendbuf' utilisée and error C4703: variable de pointeur locale potentiellement non initialisée 'tab_indice' utilisée ) when I execute this program: > #include <stdio.h> > #include <stdlib.h> > #include <mpi.h> > > #define DIMENSION 10 > > int main(int argc, char** argv) > { > int myrank, i, n_procs; > int * sendbuf; //buffer à disperser > int * tab_indice; //indice de début de chaque subdivision > int * tab_taille; //nombre d'éléments à envoyer > // pour chaque processus > int * rbuf; //buffer de reception > int taille; //taille de la partie reçue > > //Initialisation de MPI > MPI_Init(&argc, &argv); > MPI_Comm_size(MPI_COMM_WORLD, &n_procs); > MPI_Comm_rank(MPI_COMM_WORLD, &myrank); > > if (myrank == 0) > { > //allocation mémoire > sendbuf = (int *)malloc(n_procs*DIMENSION*sizeof(int)); > tab_indice = (int *)malloc(n_procs*sizeof(int)); > tab_taille = (int *)malloc(n_procs*sizeof(int)); > //remplissage du buffer à disperser > printf("la taille de la boucle %d \n", n_procs*DIMENSION); > for (i = 0; i<n_procs*DIMENSION; i++) sendbuf[i] = i; > //initialisation des subdivisions > for (i = 0; i<n_procs; i++) > { > //nombre d'éléments à envoyer > tab_taille[i] = DIMENSION - i; > printf("tab_taille[%d]=%d \n", i, tab_taille[i]); > //indice de début du processus i > // = celui de i-1 + nombre d'éléments > // envoyés par i-1 > if (i != 0) tab_indice[i] = tab_indice[i - 1] + tab_taille[i - 1]; > else tab_indice[i] = 0; > printf("tab_indice[%d]=%d \n", i, tab_indice[i]); > } > } > //communication de la taille de la partie reçue à chaque processus > MPI_Scatter(tab_taille, 1, MPI_INT, &taille, 1, MPI_INT, 0, MPI_COMM_WORLD); > //allocation du buffer de reception > rbuf = (int *)malloc(taille*sizeof(int)); > //dispersion > MPI_Scatterv(sendbuf, tab_taille, tab_indice, MPI_INT, rbuf,DIMENSION, MPI_INT, 0, MPI_COMM_WORLD); > //affichage > printf("processus %d : [ ", myrank); > for (i = 0; i<taille; i++) printf("%d ", rbuf[i]); > printf("]\n"); > > //desallocation mémoire > free(rbuf); > if (myrank == 0) { > free(sendbuf); > free(tab_indice); > free(tab_taille); > } > > MPI_Finalize(); > exit(EXIT_SUCCESS); > } > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Pavan Balaji ✉️ http://www.mcs.anl.gov/~balaji _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

(no subject)
by Chafik sanaa 11 Nov '14

11 Nov '14

Hi, What should I do? i have the folowing error (error C4703: variable de pointeur locale potentiellement non initialisée 'sendbuf' utilisée and error C4703: variable de pointeur locale potentiellement non initialisée 'tab_indice' utilisée ) when I execute this program: #include <stdio.h> #include <stdlib.h> #include <mpi.h> #define DIMENSION 10 int main(int argc, char** argv) { int myrank, i, n_procs; int * sendbuf; //buffer à disperser int * tab_indice; //indice de début de chaque subdivision int * tab_taille; //nombre d'éléments à envoyer // pour chaque processus int * rbuf; //buffer de reception int taille; //taille de la partie reçue //Initialisation de MPI MPI_Init(&argc, &argv); MPI_Comm_size(MPI_COMM_WORLD, &n_procs); MPI_Comm_rank(MPI_COMM_WORLD, &myrank); if (myrank == 0) { //allocation mémoire sendbuf = (int *)malloc(n_procs*DIMENSION*sizeof(int)); tab_indice = (int *)malloc(n_procs*sizeof(int)); tab_taille = (int *)malloc(n_procs*sizeof(int)); //remplissage du buffer à disperser printf("la taille de la boucle %d \n", n_procs*DIMENSION); for (i = 0; i<n_procs*DIMENSION; i++) sendbuf[i] = i; //initialisation des subdivisions for (i = 0; i<n_procs; i++) { //nombre d'éléments à envoyer tab_taille[i] = DIMENSION - i; printf("tab_taille[%d]=%d \n", i, tab_taille[i]); //indice de début du processus i // = celui de i-1 + nombre d'éléments // envoyés par i-1 if (i != 0) tab_indice[i] = tab_indice[i - 1] + tab_taille[i - 1]; else tab_indice[i] = 0; printf("tab_indice[%d]=%d \n", i, tab_indice[i]); } } //communication de la taille de la partie reçue à chaque processus MPI_Scatter(tab_taille, 1, MPI_INT, &taille, 1, MPI_INT, 0, MPI_COMM_WORLD); //allocation du buffer de reception rbuf = (int *)malloc(taille*sizeof(int)); //dispersion MPI_Scatterv(sendbuf, tab_taille, tab_indice, MPI_INT, rbuf,DIMENSION, MPI_INT, 0, MPI_COMM_WORLD); //affichage printf("processus %d : [ ", myrank); for (i = 0; i<taille; i++) printf("%d ", rbuf[i]); printf("]\n"); //desallocation mémoire free(rbuf); if (myrank == 0) { free(sendbuf); free(tab_indice); free(tab_taille); } MPI_Finalize(); exit(EXIT_SUCCESS); } _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Assertion failed in file src/mpid/ch3/src/ch3u_handle_send_req.c at line 61 (RMA && Derived datatypes)&In-Reply-To=<0A407957589BAB4F924824150C4293EF588072EC@CITESMBX2.ad.uillinois.edu>
by Balaji, Pavan 11 Nov '14

11 Nov '14

Victor, You don’t need admin access for installing MXM. You can just download it from here: http://www.mellanox.com/page/products_dyn?product_family=189&mtag=hpc-x Simply untar it in your home directory, and you can build mpich by pointing to that location with --with-mxm=/path/to/mxm FWIW, the ch3:nemesis:ib code is now fixed in mpich/master. You should be able to use it from tonight’s tarball as well. — Pavan > On Nov 11, 2014, at 7:18 AM, Victor Vysotskiy <victor.vysotskiy(a)teokem.lu.se> wrote: > > Dear Xin, > > unfortunately, MXM is not avaiable on our cluster and admins won't install it. So, should I wait untill you repair 'ch3:nemesis:ib'? Actually, I would be happy even if my test-code starts working on my laptop :) > > Best regards, > Victor > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss -- Pavan Balaji ✉️ http://www.mcs.anl.gov/~balaji _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Assertion failed in file src/mpid/ch3/src/ch3u_handle_send_req.c at line 61 (RMA && Derived datatypes)&In-Reply-To=<0A407957589BAB4F924824150C4293EF588072EC@CITESMBX2.ad.uillinois.edu>
by Victor Vysotskiy 11 Nov '14

11 Nov '14

Dear Xin, unfortunately, MXM is not avaiable on our cluster and admins won't install it. So, should I wait untill you repair 'ch3:nemesis:ib'? Actually, I would be happy even if my test-code starts working on my laptop :) Best regards, Victor _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] (no subject)
by Rajeev Thakur 10 Nov '14

10 Nov '14

Try MPI_Scatterv. On Nov 10, 2014, at 4:11 PM, Chafik sanaa <san.chafik(a)gmail.com> wrote: > if I want to distribute data (as vectors) with different sizes to each process, which function I should use? > _______________________________________________ > discuss mailing list discuss(a)mpich.org > To manage subscription options or unsubscribe: > https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

(no subject)
by Chafik sanaa 10 Nov '14

10 Nov '14

if I want to distribute data (as vectors) with different sizes to each process, which function I should use? _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Assertion failed in file src/mpid/ch3/src/ch3u_handle_send_req.c at line 61 (RMA && Derived datatypes)
by Zhao, Xin 10 Nov '14

10 Nov '14

Hi Victor, I will look into your assertion failure code. The RMA part in ch3:nemesis:ib is broken. Could you try to use MXM netmod by ch3:nemesis:mxm? Xin ________________________________________ From: Victor Vysotskiy [victor.vysotskiy(a)teokem.lu.se] Sent: Monday, November 10, 2014 1:02 PM To: discuss(a)mpich.org Subject: Re: [mpich-discuss] Assertion failed in file src/mpid/ch3/src/ch3u_handle_send_req.c at line 61 (RMA && Derived datatypes) Dear Pavan, well, the problem still remains with the latest ('e3e7d765e') mpich nightly snapshot cloned from git://git.mpich.org/mpich.git. There is an another compiling problem when MPICH is configure with the '--with-device=ch3:nemesis:ib' option: src/mpid/ch3/channels/nemesis/netmod/ib/ib_poll.c(37): error: expected a declaration } ^ src/mpid/ch3/channels/nemesis/netmod/ib/ib_poll.c(89): warning #12: parsing restarts here after previous syntax error goto fn_exit; ^ src/mpid/ch3/channels/nemesis/netmod/ib/ib_poll.c(90): error: expected a declaration } ^ src/mpid/ch3/channels/nemesis/netmod/ib/ib_poll.c(97): warning #12: parsing restarts here after previous syntax error ibv_poll_cq(MPID_nem_ib_rc_shared_scq, /*3 */ MPID_NEM_IB_COM_MAX_CQ_HEIGHT_DRAIN, &cqe[0]); ^ src/mpid/ch3/channels/nemesis/netmod/ib/ib_poll.c(100): error: expected a declaration MPIU_ERR_CHKANDJUMP(result < 0, mpi_errno, MPI_ERR_OTHER, "**netmod,ib,ibv_poll_cq"); ^ src/mpid/ch3/channels/nemesis/netmod/ib/ib_poll.c(102): error: expected a declaration if (result > 0) { ^ src/mpid/ch3/channels/nemesis/netmod/ib/ib_poll.c(126): warning #12: parsing restarts here after previous syntax error msg_type = MPIDI_Request_get_msg_type(req); Could you please have a look at the issues? With best regards, Victor. _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0

Re: [mpich-discuss] Assertion failed in file src/mpid/ch3/src/ch3u_handle_send_req.c at line 61 (RMA && Derived datatypes)
by Victor Vysotskiy 10 Nov '14

10 Nov '14

Dear Pavan, well, the problem still remains with the latest ('e3e7d765e') mpich nightly snapshot cloned from git://git.mpich.org/mpich.git. There is an another compiling problem when MPICH is configure with the '--with-device=ch3:nemesis:ib' option: src/mpid/ch3/channels/nemesis/netmod/ib/ib_poll.c(37): error: expected a declaration } ^ src/mpid/ch3/channels/nemesis/netmod/ib/ib_poll.c(89): warning #12: parsing restarts here after previous syntax error goto fn_exit; ^ src/mpid/ch3/channels/nemesis/netmod/ib/ib_poll.c(90): error: expected a declaration } ^ src/mpid/ch3/channels/nemesis/netmod/ib/ib_poll.c(97): warning #12: parsing restarts here after previous syntax error ibv_poll_cq(MPID_nem_ib_rc_shared_scq, /*3 */ MPID_NEM_IB_COM_MAX_CQ_HEIGHT_DRAIN, &cqe[0]); ^ src/mpid/ch3/channels/nemesis/netmod/ib/ib_poll.c(100): error: expected a declaration MPIU_ERR_CHKANDJUMP(result < 0, mpi_errno, MPI_ERR_OTHER, "**netmod,ib,ibv_poll_cq"); ^ src/mpid/ch3/channels/nemesis/netmod/ib/ib_poll.c(102): error: expected a declaration if (result > 0) { ^ src/mpid/ch3/channels/nemesis/netmod/ib/ib_poll.c(126): warning #12: parsing restarts here after previous syntax error msg_type = MPIDI_Request_get_msg_type(req); Could you please have a look at the issues? With best regards, Victor. _______________________________________________ discuss mailing list discuss(a)mpich.org To manage subscription options or unsubscribe: https://lists.mpich.org/mailman/listinfo/discuss

1 0