January 2018 - Nek5000-users - mailman3.cels.anl.gov

3D v/s. 2D computation
by nek5000-users＠lists.mcs.anl.gov 14 Mar '18

14 Mar '18

Dear All, I am carrying out 3D simulations using NEK5000 in a hexahedral domain. Symmetry boundary conditions have been imposed on the side wall and stress free conditions are prescribed on the top and bottom boundary. At the inlet and outlet, Dirichlet and stress free conditions are used, respectively. I observe that the flow on the symmetry plane is different from that obtained via 2D simulation. Is this a numerical artifact (increased diffusion because of 3D mesh) or is it physical? If I increase the resolution of the mesh in third dimension, do I expect to get the 2D simulation results on the symmetry plane? Any input will be appreciated. Many thanks, NN.

1 2

'Divergence' using low-Mach number solver
by nek5000-users＠lists.mcs.anl.gov 31 Jan '18

31 Jan '18

Hello everyone, I'm trying to use low Mach number solver to simulate flow entering a domain at a stipulated Reynolds number. The boundary conditions are specified in the user file. When Im trying to run the case, it shows 'DIVERGENCE' as shown in the image attached with this email. The files associated with the simulation are also attached. Thanks for the help in advance. -- Nitish Kovalam Aerospace Engineering Indian Institute of Space Science and Technology

1 1

Restart statistics from f00*
by nek5000-users＠lists.mcs.anl.gov 29 Jan '18

29 Jan '18

Hi, Neks! I wonder if there is any way to restart the statistics from my f000* files? Don't mean base flow f000*, I'm talking about f000* from my outpost in .usr file (ex. st1_pipe0.f00001), where I store my arrays for different correlations, etc. I've read in this mailing list about load_fld subroutine, but it is still not clear for me, could I use it for my purpose? I will be glad to hear your options and comments. Best regards, Markus

1 3

Coupling between velocity and temperature fields
by nek5000-users＠lists.mcs.anl.gov 26 Jan '18

26 Jan '18

Hi Neks I i`m trying to add heat transfer to an isothermal DNS case I have been working on. After setting the boundary conditions types in usrdat2() and the initial and boundary values in usrbc() and usric() I turned set: T IFFLOW T IFTRAN T T IFADVC (one entry per field) T IFHEAT T IFCHAR and T COORDINATES T VELOCITY T PRESSURE T TEMPERATURE F TEMPERATURE GRADIENT The simulation is restarted from a previous field file and the velocity and pressure fields are loaded as restart options. When the simulation runs, the temperature fields seems to be diffusing through a stationary medium, even though my velocity field is developed. Are there any switches or parameters that i might have overlooked? Regards

1 0

auto_averager(rm2 files)
by nek5000-users＠lists.mcs.anl.gov 26 Jan '18

26 Jan '18

Hi Nek, Is there anyone know after using auto_averager(rm2 files) which variables are returned as mean(vx*vy) mean(vx*vz), and mean(vy*vz)? Kind regards, Jian

1 0

Re: [Nek5000-users] turbChannel compiling issue after adding more elements
by nek5000-users＠lists.mcs.anl.gov 25 Jan '18

25 Jan '18

The local problem size (defined by the local number of elements) is important when it comes to time to solution and memory consumption. Typically the 2GB limit is reached for lx1=8 and 500-1000 elements. Yes, this limit can be relaxed by e.g. -mcmodel=medium but this is not needed. The 2GB limit is not a real limitation as in almost all production runs the number of elements per MPI rank is way lower (reasonable time to solution). In your case you compiled Nek5000 for a local problem size of lelg/lpmin = 36864 elements. This is a pretty huge number! No way this fits into the 2GB memory limit. Moreoever, your computation would take forever if you would run Nek5000 with just lpmin MPI ranks. Hope this helps, Stefan -----Original message----- > From:[email protected] <nek5000-users(a)lists.mcs.anl.gov> > Sent: Thursday 25th January 2018 18:21 > To: nek5000-users(a)lists.mcs.anl.gov > Subject: Re: [Nek5000-users] turbChannel compiling issue after adding more elements > > You need a way larger lpmin for 73728 elements say 128! > > On 25 Jan 2018, at 18:05, "nek5000-users(a)lists.mcs.anl.gov <mailto:[email protected]>" <nek5000-users(a)lists.mcs.anl.gov <mailto:[email protected]>> wrote: > > Hello, > Thank you all for your suggestions. However they did not really work either. The current work on the turbulent channel is in fact based on the original turbChannel in the example. The current > lelg number is the actual number of elements (73728) I used. I also tried to increase the > lelg a bit to 73750, which did not make any difference. I decreased lpmin from 4 to 2 and unfortunately this did not work either. I wonder if there is any actual limitation to the number of elements that shall be used provided the physical dimentions > defined in the code (2 pi delta * pi delta * 2 delta). > Thank you all for your time and suggestions! > Regards, > Ming > _______________________________________________ > Nek5000-users mailing list > Nek5000-users(a)lists.mcs.anl.gov <mailto:[email protected]> > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users <https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users> > _______________________________________________ > Nek5000-users mailing list > Nek5000-users(a)lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users

1 0

Re: [Nek5000-users] turbChannel compiling issue after adding more elements
by nek5000-users＠lists.mcs.anl.gov 25 Jan '18

25 Jan '18

Hello, Thank you all for your suggestions. However they did not really work either. The current work on the turbulent channel is in fact based on the original turbChannel in the example. The current lelg number is the actual number of elements (73728) I used. I also tried to increase the lelg a bit to 73750, which did not make any difference. I decreased lpmin from 4 to 2 and unfortunately this did not work either. I wonder if there is any actual limitation to the number of elements that shall be used provided the physical dimentions defined in the code (2 pi delta * pi delta * 2 delta). Thank you all for your time and suggestions! Regards, Ming

1 1

Re: [Nek5000-users] turbChannel compiling issue after adding more elements
by nek5000-users＠lists.mcs.anl.gov 25 Jan '18

25 Jan '18

Typically this is not needed. I guess you problem is related to some parameters in SIZE: - set lelg to the total number of elements of your mesh - set lpmin to a reasonable (small) value -----Original message----- > From:[email protected] <nek5000-users(a)lists.mcs.anl.gov> > Sent: Wednesday 24th January 2018 23:24 > To: nek5000-users(a)lists.mcs.anl.gov > Subject: Re: [Nek5000-users] turbChannel compiling issue after adding more elements > > Hi Ming, > > This usually happens to me when the resultant executable requires more memory allocation than available. If you haven't already, you can add > > -mcmodel=medium > > to the FFLAGS variable in makenek. This tells the compiler that it can create an executable that requires more than 2 GB of memory. Otherwise, if you already have that in there, you might be running into the limit of available physical memory of your system. > > Sincerely, > Kento > > > On Jan 24, 2018, at 15:07, nek5000-users(a)lists.mcs.anl.gov wrote: > > > > Dear Nek team, > > > > I ran into an odd issue when compiling the file after adding more elements. Number of elements have been increased to 48(x)*32(y)*48(z) from the default 16(x)*12(y)*8(z). Everything went very smooth (genbox and genmap both worked well) till the last step in “makenek” when it started to generate .ffile; error occurred and it says: > > > > obj/turbChannel.o: In function `uservp_': > > turbChannel.f:(.text+0x17): relocation truncated to fit: R_X86_64_32S against symbol `hcglb_' defined in COMMON section in obj/turbChannel.o > > turbChannel.f:(.text+0x30): relocation truncated to fit: R_X86_64_PC32 against symbol `nekuse_' defined in COMMON section in obj/turbChannel.o > > turbChannel.f:(.text+0x3c): relocation truncated to fit: R_X86_64_32S against symbol `cdsmag_' defined in COMMON section in obj/turbChannel.o > > turbChannel.f:(.text+0x43): relocation truncated to fit: R_X86_64_PC32 against symbol `nekuse_' defined in COMMON section in obj/turbChannel.o > > obj/turbChannel.o: In function `userf_': > > turbChannel.f:(.text+0x56): relocation truncated to fit: R_X86_64_PC32 against symbol `cforce_' defined in COMMON section in obj/turbChannel.o > > turbChannel.f:(.text+0x5d): relocation truncated to fit: R_X86_64_PC32 against symbol `nekuse_' defined in COMMON section in obj/turbChannel.o > > turbChannel.f:(.text+0x64): relocation truncated to fit: R_X86_64_PC32 against symbol `nekuse_' defined in COMMON section in obj/turbChannel.o > > turbChannel.f:(.text+0x6b): relocation truncated to fit: R_X86_64_PC32 against symbol `nekuse_' defined in COMMON section in obj/turbChannel.o > > obj/turbChannel.o: In function `userq_': > > turbChannel.f:(.text+0x73): relocation truncated to fit: R_X86_64_PC32 against symbol `nekuse_' defined in COMMON section in obj/turbChannel.o > > obj/turbChannel.o: In function `userchk_': > > turbChannel.f:(.text+0x93): relocation truncated to fit: R_X86_64_PC32 against symbol `dimn_' defined in COMMON section in obj/turbChannel.o > > turbChannel.f:(.text+0x9b): additional relocation overflows omitted from the output > > make: *** [makefile:120: nek5000] Error 1 > > > > Such error did not occur when running the default mesh settings. Has anyone seen this before? Any suggestions? > > > > Thanks a lot for the help! > > > > Regards, > > > > Ming > > _______________________________________________ > > Nek5000-users mailing list > > Nek5000-users(a)lists.mcs.anl.gov > > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users(a)lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users >

1 0

Re: [Nek5000-users] turbChannel compiling issue after adding more elements
by nek5000-users＠lists.mcs.anl.gov 25 Jan '18

25 Jan '18

Are you using the mesh from the example otherwise you have to create new AMG files for your mesh. See here: https://lists.mcs.anl.gov/mailman/htdig/nek5000-users/2017-December/004854.… -----Original message----- > From:[email protected] <nek5000-users(a)lists.mcs.anl.gov> > Sent: Thursday 25th January 2018 5:58 > To: nek5000-users(a)lists.mcs.anl.gov > Subject: Re: [Nek5000-users] turbChannel compiling issue after adding more elements > > Hi Ming & everyone, > > I tracked down the error to a part of the gslib code (Nek5000/3rd_party/gslib/gslib-1.0.1/src/amg.c:487). Unfortunately, I am not familiar with AMG or gslib code so I cannot help you with this problem. > > If someone is familiar with this issue, please help Ming out. > > Sincerely, > Kento > > > On Jan 24, 2018, at 22:40, nek5000-users(a)lists.mcs.anl.gov wrote: > > > > Hello Kento, > > > > Thank you for your help! The compiler now works. But I ran into another issue; after it started to run, it soon stopped and reported an error saying: > > > > AMG: preading through rows ... done > > ERROR (proc 0000, src/amg.c:487): AMG: missing data for some rows > > > > an error occured: dying … > > > > Would you run into this issue earlier before? > > > > Again thanks a lot for the suggestions. > > > > Regards, > > > > Ming > > > > *********** complete logfile ********* > > Reading /global/home/hpc4076/Nek5000/run/turbChannel/turbChannel.par > > general = [(null)] > > general:stopat = [endTime] > > general:endtime = [200] > > general:dt = [0] > > general:timestepper = [bdf2] > > general:extrapolation = [OIFS] > > general:variabledt = [yes] > > general:targetcfl = [2.0] > > general:writecontrol = [runTime] > > general:writeinterval = [200.0] > > general:userparam01 = [100] > > general:userparam02 = [20] > > general:userparam03 = [1] > > general:filtering = [hpfrt] > > general:filterweight = [10] > > general:filtercutoffratio = [0.9] > > problemtype = [(null)] > > problemtype:variableproperties = [no] > > problemtype:equation = [incompNS] > > pressure = [(null)] > > pressure:preconditioner = [semg_amg] > > pressure:residualtol = [1e-04] > > pressure:residualproj = [yes] > > velocity = [(null)] > > velocity:residualtol = [1e-07] > > velocity:density = [1] > > velocity:viscosity = [-10900] > > Reading /global/home/hpc4076/Nek5000/run/turbChannel/turbChannel.re2 > > mapping elements to processors > > Reading /global/home/hpc4076/Nek5000/run/turbChannel/turbChannel.ma2 > > 0 9216 9216 73728 73728 NELV > > 5 9216 9216 73728 73728 NELV > > 7 9216 9216 73728 73728 NELV > > 1 9216 9216 73728 73728 NELV > > 3 9216 9216 73728 73728 NELV > > 6 9216 9216 73728 73728 NELV > > 4 9216 9216 73728 73728 NELV > > 2 9216 9216 73728 73728 NELV > > > > element load imbalance: 0 9216 9216 > > done :: mapping 0.66962E-01 sec > > > > > > preading mesh > > preading bc for ifld 1 > > done :: read .re2 file 0.30 sec > > > > nelgt/nelgv/lelt: 73728 73728 18439 > > lx1 /lx2 /lx3 : 8 8 8 > > setup mesh topology > > Right-handed check complete for 73728 elements. OK. > > setvert3d: 8 9476352 25401600 9476352 9476352 > > call usrsetvert > > done :: usrsetvert > > > > gs_setup: 413952 unique labels shared > > pairwise times (avg, min, max): 0.000140989 0.000133355 0.000146356 > > crystal router : 0.0012817 0.00128028 0.00128291 > > used all_to_all method: pairwise > > handle bytes (avg, min, max): 3.26526e+07 32652644 32652644 > > buffer bytes (avg, min, max): 1.68269e+06 1682688 1682688 > > setupds time 1.9997E+00 seconds 0 8 9476352 73728 > > 8 max multiplicity > > done :: setup mesh topology > > > > call usrdat > > done :: usrdat > > > > generate geometry data > > NOTE: All elements deformed , param(59) ^=0 > > done :: generate geometry data > > > > call usrdat2 > > done :: usrdat2 > > > > regenerate geometry data 1 > > NOTE: All elements deformed , param(59) ^=0 > > done :: regenerate geometry data 1 > > > > verify mesh topology > > 0.000000000000000E+000 6.28318530717959 Xrange > > -1.00000000000000 1.00000000000000 Yrange > > 0.000000000000000E+000 3.14159265358979 Zrange > > done :: verify mesh topology > > > > IFTRAN = T > > IFFLOW = T > > IFHEAT = F > > IFSPLIT = T > > IFLOMACH = F > > IFUSERVP = F > > IFUSERMV = F > > IFPERT = F > > IFADJ = F > > IFSTRS = F > > IFCHAR = T > > IFCYCLIC = F > > IFAXIS = F > > IFMVBD = F > > IFMELT = F > > IFNEKNEK = F > > IFSYNC = F > > > > IFVCOR = T > > IFINTQ = F > > IFGEOM = F > > IFSURT = F > > IFWCNO = F > > > > IFTMSH for field 1 = F > > IFADVC for field 1 = T > > IFNONL for field 1 = F > > > > Dealiasing enabled, lxd= 8 > > > > Estimated eigenvalues > > EIGAA = 1.23913370009078 > > EIGGA = 30588693.2337829 > > EIGAE = 0.250000000000000 > > EIGAS = 1.681282119258565E-002 > > EIGGE = 30588693.2337829 > > EIGGS = 2.00000000000000 > > > > verify mesh topology > > 0.000000000000000E+000 6.28318530717959 Xrange > > -1.00000000000000 1.00000000000000 Yrange > > 0.000000000000000E+000 3.14159265358979 Zrange > > done :: verify mesh topology > > > > E-solver strategy: 0 itr > > mg_nx: 1 3 7 > > mg_ny: 1 3 7 > > mg_nz: 1 3 7 > > call usrsetvert > > done :: usrsetvert > > > > gs_setup: 8448 unique labels shared > > pairwise times (avg, min, max): 1.21617e-05 1.1921e-05 1.24227e-05 > > crystal router : 3.65702e-05 3.65344e-05 3.66269e-05 > > all reduce : 0.00014386 0.000142943 0.000144376 > > used all_to_all method: pairwise > > handle bytes (avg, min, max): 728420 728420 728420 > > buffer bytes (avg, min, max): 37632 37632 37632 > > setupds time 2.0250E-02 seconds 1 2 76032 73728 > > setvert3d: 4 1421568 2011392 1421568 1421568 > > call usrsetvert > > done :: usrsetvert > > > > gs_setup: 76032 unique labels shared > > pairwise times (avg, min, max): 4.62178e-05 4.47887e-05 4.74119e-05 > > crystal router : 0.000222897 0.000222723 0.000223086 > > all reduce : 0.000974339 0.000967893 0.000979289 > > used all_to_all method: pairwise > > handle bytes (avg, min, max): 5.91396e+06 5913956 5913956 > > buffer bytes (avg, min, max): 315648 315648 315648 > > setupds time 3.5739E-01 seconds 2 4 1421568 73728 > > setvert3d: 4 1421568 2011392 1421568 1421568 > > call usrsetvert > > done :: usrsetvert > > > > gs_setup: 76032 unique labels shared > > pairwise times (avg, min, max): 4.5131e-05 4.40137e-05 4.5883e-05 > > crystal router : 0.000217508 0.00021732 0.000217639 > > all reduce : 0.000978618 0.000972797 0.000983269 > > used all_to_all method: pairwise > > handle bytes (avg, min, max): 5.91396e+06 5913956 5913956 > > buffer bytes (avg, min, max): 315648 315648 315648 > > setupds time 3.5094E-01 seconds 3 4 1421568 73728 > > setvert3d: 6 4555008 9273600 4555008 4555008 > > call usrsetvert > > done :: usrsetvert > > > > gs_setup: 211200 unique labels shared > > pairwise times (avg, min, max): 9.59221e-05 9.2321e-05 9.80701e-05 > > crystal router : 0.000574208 0.000573776 0.000574501 > > used all_to_all method: pairwise > > handle bytes (avg, min, max): 1.65554e+07 16555364 16555364 > > buffer bytes (avg, min, max): 864000 864000 864000 > > setupds time 9.2152E-01 seconds 4 6 4555008 73728 > > regenerate geometry data 1 > > NOTE: All elements deformed , param(59) ^=0 > > done :: regenerate geometry data 1 > > > > h1_mg_nx: 1 3 7 > > h1_mg_ny: 1 3 7 > > h1_mg_nz: 1 3 7 > > call usrsetvert > > done :: usrsetvert > > > > gs_setup: 8448 unique labels shared > > pairwise times (avg, min, max): 1.42743e-05 1.37607e-05 1.47085e-05 > > crystal router : 3.54692e-05 3.53276e-05 3.56395e-05 > > all reduce : 0.000145775 0.000145112 0.000146347 > > used all_to_all method: pairwise > > handle bytes (avg, min, max): 728420 728420 728420 > > buffer bytes (avg, min, max): 37632 37632 37632 > > setupds time 1.9277E-02 seconds 5 2 76032 73728 > > setvert3d: 4 1421568 2011392 1421568 1421568 > > call usrsetvert > > done :: usrsetvert > > > > gs_setup: 76032 unique labels shared > > pairwise times (avg, min, max): 4.74352e-05 4.61341e-05 4.8379e-05 > > crystal router : 0.000219904 0.000219596 0.000220055 > > all reduce : 0.00108293 0.00107907 0.00108614 > > used all_to_all method: pairwise > > handle bytes (avg, min, max): 5.91396e+06 5913956 5913956 > > buffer bytes (avg, min, max): 315648 315648 315648 > > setupds time 3.5849E-01 seconds 6 4 1421568 73728 > > setvert3d: 4 1421568 2011392 1421568 1421568 > > call usrsetvert > > done :: usrsetvert > > > > gs_setup: 76032 unique labels shared > > pairwise times (avg, min, max): 4.99966e-05 4.93097e-05 5.04928e-05 > > crystal router : 0.000220982 0.000220777 0.000221201 > > all reduce : 0.000975639 0.000969762 0.000980587 > > used all_to_all method: pairwise > > handle bytes (avg, min, max): 5.91396e+06 5913956 5913956 > > buffer bytes (avg, min, max): 315648 315648 315648 > > setupds time 3.5221E-01 seconds 7 4 1421568 73728 > > setvert3d: 6 4555008 9273600 4555008 4555008 > > call usrsetvert > > done :: usrsetvert > > > > gs_setup: 211200 unique labels shared > > pairwise times (avg, min, max): 9.7033e-05 9.53629e-05 9.89481e-05 > > crystal router : 0.000575673 0.000575274 0.00057601 > > used all_to_all method: pairwise > > handle bytes (avg, min, max): 1.65554e+07 16555364 16555364 > > buffer bytes (avg, min, max): 864000 864000 864000 > > setupds time 9.0969E-01 seconds 8 6 4555008 73728 > > setvert3d: 8 9476352 25401600 9476352 9476352 > > call usrsetvert > > done :: usrsetvert > > > > gs_setup: 413952 unique labels shared > > pairwise times (avg, min, max): 0.000145018 0.00014225 0.000147666 > > crystal router : 0.00127508 0.00127457 0.00127577 > > used all_to_all method: pairwise > > handle bytes (avg, min, max): 3.26526e+07 32652644 32652644 > > buffer bytes (avg, min, max): 1.68269e+06 1682688 1682688 > > setupds time 1.9358E+00 seconds 9 8 9476352 73728 > > setvert3d: 10 16185600 53934336 16185600 16185600 > > call usrsetvert > > done :: usrsetvert > > > > gs_setup: 684288 unique labels shared > > pairwise times (avg, min, max): 0.000185176 0.000182058 0.000188558 > > crystal router : 0.00236343 0.00236093 0.00236685 > > used all_to_all method: pairwise > > handle bytes (avg, min, max): 5.42058e+07 54205796 54205796 > > buffer bytes (avg, min, max): 2.77171e+06 2771712 2771712 > > setupds time 3.2958E+00 seconds 10 10 16185600 73728 > > setup h1 coarse grid, nx_crs= 2 > > call usrsetvert > > done :: usrsetvert > > > > gs_setup: 8448 unique labels shared > > pairwise times (avg, min, max): 1.58059e-05 1.48741e-05 1.6321e-05 > > crystal router : 3.54276e-05 3.51642e-05 3.56729e-05 > > all reduce : 0.000144706 0.000143665 0.00014533 > > used all_to_all method: pairwise > > handle bytes (avg, min, max): 728420 728420 728420 > > buffer bytes (avg, min, max): 37632 37632 37632 > > gs_setup: 8448 unique labels shared > > pairwise times (avg, min, max): 1.56632e-05 1.53686e-05 1.61476e-05 > > crystal router : 1.74988e-05 1.74323e-05 1.75631e-05 > > all reduce : 0.000148094 0.00014731 0.000148594 > > used all_to_all method: pairwise > > handle bytes (avg, min, max): 368228 368228 368228 > > buffer bytes (avg, min, max): 17664 17664 17664 > > AMG version 2.01 > > AMG: 8 levels > > AMG Chebyshev smoother data: > > AMG level 1: 3 iterations with rho = 0.79256 > > AMG level 2: 2 iterations with rho = 0.668297 > > AMG level 3: 2 iterations with rho = 0.481307 > > AMG level 4: 2 iterations with rho = 0.409707 > > AMG level 5: 2 iterations with rho = 0.458605 > > AMG level 6: 3 iterations with rho = 0.696293 > > AMG level 7: 2 iterations with rho = 0.666743 > > AMG: 1664 rows > > AMG: preading through rows ... done > > ERROR (proc 0000, src/amg.c:487): AMG: missing data for some rows > > > > an error occured: dying ... > > _______________________________________________ > > Nek5000-users mailing list > > Nek5000-users(a)lists.mcs.anl.gov > > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users > > _______________________________________________ > Nek5000-users mailing list > Nek5000-users(a)lists.mcs.anl.gov > https://lists.mcs.anl.gov/mailman/listinfo/nek5000-users >

1 0

Re: [Nek5000-users] turbChannel compiling issue after adding more elements
by nek5000-users＠lists.mcs.anl.gov 24 Jan '18

24 Jan '18

Hello Kento, Thank you for your help! The compiler now works. But I ran into another issue; after it started to run, it soon stopped and reported an error saying: AMG: preading through rows ... done ERROR (proc 0000, src/amg.c:487): AMG: missing data for some rows an error occured: dying … Would you run into this issue earlier before? Again thanks a lot for the suggestions. Regards, Ming *********** complete logfile ********* ________________________________ Reading /global/home/hpc4076/Nek5000/run/turbChannel/turbChannel.par general = [(null)] general:stopat = [endTime] general:endtime = [200] general:dt = [0] general:timestepper = [bdf2] general:extrapolation = [OIFS] general:variabledt = [yes] general:targetcfl = [2.0] general:writecontrol = [runTime] general:writeinterval = [200.0] general:userparam01 = [100] general:userparam02 = [20] general:userparam03 = [1] general:filtering = [hpfrt] general:filterweight = [10] general:filtercutoffratio = [0.9] problemtype = [(null)] problemtype:variableproperties = [no] problemtype:equation = [incompNS] pressure = [(null)] pressure:preconditioner = [semg_amg] pressure:residualtol = [1e-04] pressure:residualproj = [yes] velocity = [(null)] velocity:residualtol = [1e-07] velocity:density = [1] velocity:viscosity = [-10900] Reading /global/home/hpc4076/Nek5000/run/turbChannel/turbChannel.re2 mapping elements to processors Reading /global/home/hpc4076/Nek5000/run/turbChannel/turbChannel.ma2 0 9216 9216 73728 73728 NELV 5 9216 9216 73728 73728 NELV 7 9216 9216 73728 73728 NELV 1 9216 9216 73728 73728 NELV 3 9216 9216 73728 73728 NELV 6 9216 9216 73728 73728 NELV 4 9216 9216 73728 73728 NELV 2 9216 9216 73728 73728 NELV element load imbalance: 0 9216 9216 done :: mapping 0.66962E-01 sec preading mesh preading bc for ifld 1 done :: read .re2 file 0.30 sec nelgt/nelgv/lelt: 73728 73728 18439 lx1 /lx2 /lx3 : 8 8 8 setup mesh topology Right-handed check complete for 73728 elements. OK. setvert3d: 8 9476352 25401600 9476352 9476352 call usrsetvert done :: usrsetvert gs_setup: 413952 unique labels shared pairwise times (avg, min, max): 0.000140989 0.000133355 0.000146356 crystal router : 0.0012817 0.00128028 0.00128291 used all_to_all method: pairwise handle bytes (avg, min, max): 3.26526e+07 32652644 32652644 buffer bytes (avg, min, max): 1.68269e+06 1682688 1682688 setupds time 1.9997E+00 seconds 0 8 9476352 73728 8 max multiplicity done :: setup mesh topology call usrdat done :: usrdat generate geometry data NOTE: All elements deformed , param(59) ^=0 done :: generate geometry data call usrdat2 done :: usrdat2 regenerate geometry data 1 NOTE: All elements deformed , param(59) ^=0 done :: regenerate geometry data 1 verify mesh topology 0.000000000000000E+000 6.28318530717959 Xrange -1.00000000000000 1.00000000000000 Yrange 0.000000000000000E+000 3.14159265358979 Zrange done :: verify mesh topology IFTRAN = T IFFLOW = T IFHEAT = F IFSPLIT = T IFLOMACH = F IFUSERVP = F IFUSERMV = F IFPERT = F IFADJ = F IFSTRS = F IFCHAR = T IFCYCLIC = F IFAXIS = F IFMVBD = F IFMELT = F IFNEKNEK = F IFSYNC = F IFVCOR = T IFINTQ = F IFGEOM = F IFSURT = F IFWCNO = F IFTMSH for field 1 = F IFADVC for field 1 = T IFNONL for field 1 = F Dealiasing enabled, lxd= 8 Estimated eigenvalues EIGAA = 1.23913370009078 EIGGA = 30588693.2337829 EIGAE = 0.250000000000000 EIGAS = 1.681282119258565E-002 EIGGE = 30588693.2337829 EIGGS = 2.00000000000000 verify mesh topology 0.000000000000000E+000 6.28318530717959 Xrange -1.00000000000000 1.00000000000000 Yrange 0.000000000000000E+000 3.14159265358979 Zrange done :: verify mesh topology E-solver strategy: 0 itr mg_nx: 1 3 7 mg_ny: 1 3 7 mg_nz: 1 3 7 call usrsetvert done :: usrsetvert gs_setup: 8448 unique labels shared pairwise times (avg, min, max): 1.21617e-05 1.1921e-05 1.24227e-05 crystal router : 3.65702e-05 3.65344e-05 3.66269e-05 all reduce : 0.00014386 0.000142943 0.000144376 used all_to_all method: pairwise handle bytes (avg, min, max): 728420 728420 728420 buffer bytes (avg, min, max): 37632 37632 37632 setupds time 2.0250E-02 seconds 1 2 76032 73728 setvert3d: 4 1421568 2011392 1421568 1421568 call usrsetvert done :: usrsetvert gs_setup: 76032 unique labels shared pairwise times (avg, min, max): 4.62178e-05 4.47887e-05 4.74119e-05 crystal router : 0.000222897 0.000222723 0.000223086 all reduce : 0.000974339 0.000967893 0.000979289 used all_to_all method: pairwise handle bytes (avg, min, max): 5.91396e+06 5913956 5913956 buffer bytes (avg, min, max): 315648 315648 315648 setupds time 3.5739E-01 seconds 2 4 1421568 73728 setvert3d: 4 1421568 2011392 1421568 1421568 call usrsetvert done :: usrsetvert gs_setup: 76032 unique labels shared pairwise times (avg, min, max): 4.5131e-05 4.40137e-05 4.5883e-05 crystal router : 0.000217508 0.00021732 0.000217639 all reduce : 0.000978618 0.000972797 0.000983269 used all_to_all method: pairwise handle bytes (avg, min, max): 5.91396e+06 5913956 5913956 buffer bytes (avg, min, max): 315648 315648 315648 setupds time 3.5094E-01 seconds 3 4 1421568 73728 setvert3d: 6 4555008 9273600 4555008 4555008 call usrsetvert done :: usrsetvert gs_setup: 211200 unique labels shared pairwise times (avg, min, max): 9.59221e-05 9.2321e-05 9.80701e-05 crystal router : 0.000574208 0.000573776 0.000574501 used all_to_all method: pairwise handle bytes (avg, min, max): 1.65554e+07 16555364 16555364 buffer bytes (avg, min, max): 864000 864000 864000 setupds time 9.2152E-01 seconds 4 6 4555008 73728 regenerate geometry data 1 NOTE: All elements deformed , param(59) ^=0 done :: regenerate geometry data 1 h1_mg_nx: 1 3 7 h1_mg_ny: 1 3 7 h1_mg_nz: 1 3 7 call usrsetvert done :: usrsetvert gs_setup: 8448 unique labels shared pairwise times (avg, min, max): 1.42743e-05 1.37607e-05 1.47085e-05 crystal router : 3.54692e-05 3.53276e-05 3.56395e-05 all reduce : 0.000145775 0.000145112 0.000146347 used all_to_all method: pairwise handle bytes (avg, min, max): 728420 728420 728420 buffer bytes (avg, min, max): 37632 37632 37632 setupds time 1.9277E-02 seconds 5 2 76032 73728 setvert3d: 4 1421568 2011392 1421568 1421568 call usrsetvert done :: usrsetvert gs_setup: 76032 unique labels shared pairwise times (avg, min, max): 4.74352e-05 4.61341e-05 4.8379e-05 crystal router : 0.000219904 0.000219596 0.000220055 all reduce : 0.00108293 0.00107907 0.00108614 used all_to_all method: pairwise handle bytes (avg, min, max): 5.91396e+06 5913956 5913956 buffer bytes (avg, min, max): 315648 315648 315648 setupds time 3.5849E-01 seconds 6 4 1421568 73728 setvert3d: 4 1421568 2011392 1421568 1421568 call usrsetvert done :: usrsetvert gs_setup: 76032 unique labels shared pairwise times (avg, min, max): 4.99966e-05 4.93097e-05 5.04928e-05 crystal router : 0.000220982 0.000220777 0.000221201 all reduce : 0.000975639 0.000969762 0.000980587 used all_to_all method: pairwise handle bytes (avg, min, max): 5.91396e+06 5913956 5913956 buffer bytes (avg, min, max): 315648 315648 315648 setupds time 3.5221E-01 seconds 7 4 1421568 73728 setvert3d: 6 4555008 9273600 4555008 4555008 call usrsetvert done :: usrsetvert gs_setup: 211200 unique labels shared pairwise times (avg, min, max): 9.7033e-05 9.53629e-05 9.89481e-05 crystal router : 0.000575673 0.000575274 0.00057601 used all_to_all method: pairwise handle bytes (avg, min, max): 1.65554e+07 16555364 16555364 buffer bytes (avg, min, max): 864000 864000 864000 setupds time 9.0969E-01 seconds 8 6 4555008 73728 setvert3d: 8 9476352 25401600 9476352 9476352 call usrsetvert done :: usrsetvert gs_setup: 413952 unique labels shared pairwise times (avg, min, max): 0.000145018 0.00014225 0.000147666 crystal router : 0.00127508 0.00127457 0.00127577 used all_to_all method: pairwise handle bytes (avg, min, max): 3.26526e+07 32652644 32652644 buffer bytes (avg, min, max): 1.68269e+06 1682688 1682688 setupds time 1.9358E+00 seconds 9 8 9476352 73728 setvert3d: 10 16185600 53934336 16185600 16185600 call usrsetvert done :: usrsetvert gs_setup: 684288 unique labels shared pairwise times (avg, min, max): 0.000185176 0.000182058 0.000188558 crystal router : 0.00236343 0.00236093 0.00236685 used all_to_all method: pairwise handle bytes (avg, min, max): 5.42058e+07 54205796 54205796 buffer bytes (avg, min, max): 2.77171e+06 2771712 2771712 setupds time 3.2958E+00 seconds 10 10 16185600 73728 setup h1 coarse grid, nx_crs= 2 call usrsetvert done :: usrsetvert gs_setup: 8448 unique labels shared pairwise times (avg, min, max): 1.58059e-05 1.48741e-05 1.6321e-05 crystal router : 3.54276e-05 3.51642e-05 3.56729e-05 all reduce : 0.000144706 0.000143665 0.00014533 used all_to_all method: pairwise handle bytes (avg, min, max): 728420 728420 728420 buffer bytes (avg, min, max): 37632 37632 37632 gs_setup: 8448 unique labels shared pairwise times (avg, min, max): 1.56632e-05 1.53686e-05 1.61476e-05 crystal router : 1.74988e-05 1.74323e-05 1.75631e-05 all reduce : 0.000148094 0.00014731 0.000148594 used all_to_all method: pairwise handle bytes (avg, min, max): 368228 368228 368228 buffer bytes (avg, min, max): 17664 17664 17664 AMG version 2.01 AMG: 8 levels AMG Chebyshev smoother data: AMG level 1: 3 iterations with rho = 0.79256 AMG level 2: 2 iterations with rho = 0.668297 AMG level 3: 2 iterations with rho = 0.481307 AMG level 4: 2 iterations with rho = 0.409707 AMG level 5: 2 iterations with rho = 0.458605 AMG level 6: 3 iterations with rho = 0.696293 AMG level 7: 2 iterations with rho = 0.666743 AMG: 1664 rows AMG: preading through rows ... done ERROR (proc 0000, src/amg.c:487): AMG: missing data for some rows an error occured: dying ...

1 1