Re: [mpich-discuss] Running mpi from bash script
Hi The application is RAMS meteorological model (http://www.atmet.com) mostly build in Fortran and some C. Compile options for RAMS are |MAKE=/usr/bin/make # libraries. LIBUTILS=./utils/libutils-$(UTILS_VERSION)-opt.a # Activate appropriate parts below, comment out others. #--------------------------------------------------------------- # NCAR Graphics. #--------------------------------------------------------------- #--------------------------------------------------------------- # If you are using a real distribution of NCAR Graphics... NCARG_DIR=/usr/local/ncarg-4.4.2/lib LIBNCARG=-L$(NCARG_DIR) -lncarg -lncarg_gks -lncarg_c -L/usr/local/ncarg-4.4.2/ncarg2d/src/libncarg_gks \ -L/usr/lib/i386-linux-gnu -lpthread -ldl -lc -lgcc_eh -lX11 #--------------------------------------------------------------- #--------------------------------------------------------------- # HDF libraries #--------------------------------------------------------------- HDF5_LIBS=-L/usr/local/hdf5/lib -lhdf5 -lz -lm HDF5_INCS=/usr/local/hdf5/include HDF5_DEFS=-DH5_USE_16_API #--------------------------------------------------------------- # Machine-dependent options. #--------------------------------------------------------------- #----------------- LINUX Portland Group pgf90/gcc --------------- CMACH=PC_LINUX1 F_COMP=/opt/intel/bin/ifort F_OPTS=-O3 -FR -fpp -assume byterecl -DRAMS_SUPPORT C_COMP=/opt/intel/bin/icc C_OPTS=-O3 -DUNDERSCORE -DLITTLE LOADER=/opt/intel/bin/ifort LOADER_OPTS= LIBS= ARCHIVE=ar rs| MPI_PATH=/usr/bin PAR_INCS=-I/usr/local/include PAR_LIBS=-L/usr/local/lib/ -lmpich PAR_DEFS=-DRAMS_MPI Regarding mpich2 path I installed from Ubuntu repositories with apt-get. Previously I just removed everything I found from mpi but can't be sure there was not anything left. When running in parallel from batch I get some processes running on the master node meteo 8288 7559 0 13:03 ? 00:00:00 time /usr/bin/mpirun -n 40 -f machinefile ./rams60 -f RAMSIN.operatiu meteo 8289 8288 0 13:03 ? 00:00:00 /usr/bin/mpirun -n 40 -f machinefile ./rams60 -f RAMSIN.operatiu meteo 8295 8290 0 13:03 ? 00:00:00 ./rams60 -f RAMSIN.operatiu meteo 8296 8290 0 13:03 ? 00:00:00 ./rams60 -f RAMSIN.operatiu meteo 8297 8290 0 13:03 ? 00:00:00 ./rams60 -f RAMSIN.operatiu meteo 8298 8290 0 13:03 ? 00:00:00 ./rams60 -f RAMSIN.operatiu meteo 8299 8290 0 13:03 ? 00:00:00 ./rams60 -f RAMSIN.operatiu meteo 8300 8290 0 13:03 ? 00:00:00 ./rams60 -f RAMSIN.operatiu meteo 8301 8290 0 13:03 ? 00:00:00 ./rams60 -f RAMSIN.operatiu meteo 8302 8290 0 13:03 ? 00:00:00 ./rams60 -f RAMSIN.operatiu When running mpirun from command line processes start in all nodes. There is still another problem that could or not be related to mpich, met model returns NaN where there should be numbers (values for meteorological variables) but maybe this has a completely different origin as initial data,... Thanks again El 22/04/2014 16:41, Reuti escribi�:
Am 22.04.2014 um 09:18 schrieb Francisco Pastor:
Hi and sorry for the delay in answering. I was out on holidays since Friday.
I recompiled the appication, just changing mpich2 path. Fine.
MPI_PATH=/usr/bin PAR_INCS=-I/usr/local/include PAR_LIBS=-L/usr/local/lib/ -lmpich Is it a C application, or are you using the wrapper during compliation to add also the Fortran interface?
PAR_DEFS=-DRAMS_MPI Do you have more than one MPI library at this location? Often it's better to have a dedicated directory where you can check the used path to it instantly.
Is there any output whether the application is now running in parallel, or executing a serial run N times only?
-- Reuti
Thanks
El 17/04/2014 17:58, Reuti escribi�:
Hi,
Am 17.04.2014 um 11:35 schrieb Francisco Pastor:
The point is that I am using a program, meteorological model RAMS, that was running fine with this mpich2 version. And it runs fine from the command line.
If the system works, should I upgrade? Will it work with my program? Can I install two different versions at the same time or should I delete the older one? Is there any chance to run Well, using a new version usually means to recompile the application to gain advantage of it. Did you compile it on your own, or is this a binary you downloaded somewhere?
-- Reuti
Thanks
El 17/04/2014 11:06, Reuti escribi�:
Hi,
Am 17.04.2014 um 10:04 schrieb Francisco Pastor:
I need to daily run an mpi job from a bash script. I have prepared the script and run from the command line, this works fine. But when I try to execute at a later time with at (at -f my_script now +1 minute) the parallel processes do not start.
These are the significant parts of my script:
# path to mpi executables ruta_mpi=/usr/local/mpich2-1.0.5p4/bin This version of MPICH is quite old. Please use a more recent one, where the necessity for starting an MPD beforehand does no longer exist.
-- Reuti
# Function that checks if mpd processes are ok (checks mpdtrace output, five nodes should be up) Comprueba_MPD() { mpdtrace > log.mpd wc -l log.mpd > check.mpd check1=`awk '{ print $1 }' check.mpd` if [ $check1 -eq 5 ]; then echo "Salida de mpdtrace correcta, hosts=$check1" else echo "Salida de mpdtrace incorrecta, hosts=$check1" $ruta_mpi/mpdboot -n 5 & sleep 10 mpdtrace fi }
# Run rams60 program with mpirun time $ruta_mpi/mpirun -n 21 ./rams60 -f RAMSIN.operatiu
I am sure missing something but I can't find out what's going on.
Thanks in advance
-- -----------
-- ----------- Dr. Francisco Pastor Meteorology department, Instituto Universitario CEAM-UMH http://www.ceam.es ----------- Mendeley profile: http://www.mendeley.com/profiles/francisco-pastor1/ Google Scholar: http://scholar.google.com/citations?user=V3mmCdkAAAAJ&hl=es Researcher ID: http://www.researcherid.com/rid/B-8331-2008 Cosis profile: http://www.cosis.net/profile/francisco.pastor ----------- mail: [email protected] skype: paco.pastor.guzman ----------- Parque Tecnologico, C/ Charles R. Darwin, 14 46980 PATERNA (Valencia), Spain Tlf. 96 131 82 27 - Fax. 96 131 81 90 --------------------------------------------------------------------- Este mensaje y los ficheros anexos son confidenciales. Los mismos contienen informaci�n reservada de la empresa que no puede ser difundida. Si usted ha recibido este correo por error, tenga la amabilidad de eliminarlo de su sistema y avisar al remitente mediante reenv�o a su direcci�n electr�nica; no deber� copiar el mensaje ni divulgar su contenido a ninguna persona. Su direcci�n de correo electr�nico junto a sus datos personales forman parte de un fichero titularidad de la Fundaci�n de la Comunidad Valenciana Centro de Estudios Ambientales del Mediterr�neo - CEAM, con CIF: G-46957213, cuya finalidad es la de mantener el contacto con Ud. De acuerdo con la Ley Org�nica 15/1999, usted puede ejercitar sus derechos de acceso, rectificaci�n, cancelaci�n y, en su caso, oposici�n enviando una solicitud por escrito, acompa�ada de una fotocopia de su DNI dirigida a: Fundaci�n de la Comunidad Valenciana Centro de Estudios Ambientales del Mediterr�neo - CEAM. C/ Charles R. Darwin, 14. Parque Tecnol�gico.46980 PATERNA (Valencia). This message and the attached files are confidential. They contain reserved information belonging to our centre and are not to be broadcast. If you have received this email by mistake, please delete it from your system and alert the sender by returning it to his/her email address. You must not copy or divulge the contents of the message to anyone. Your email address and personal data are included in a file belonging to the Fundaci�n de la Comunidad Valenciana Centro de Estudios Ambientales del Mediterr�neo - CEAM, con CIF: G-46957213. The purpose of this file is to allow us to keep in contact with you. In accordance with Organic Law 15/1999, you are permitted to access, rectify, cancel or oppose the contents of this file by submitting a written request, accompanied by a photocopy of your DNI, to: Fundaci�n de la Comunidad Valenciana Centro de Estudios Ambientales del Mediterr�neo - CEAM. C/ Charles R. Darwin, 14. Parque Tecnol�gico.46980 PATERNA (Valencia).
participants (1)
-
Francisco Pastor