Please ./create_test with "-j 1" on Titan
?Hi all, There have been some problems with "./create_test" on Titan eating up too many resources on the external login nodes, and admins have had to kill some of the jobs. I assume this is because of the build phase, so for now, please run ./create_test with "-j 1" (see ./create_test --help) when running on Titan to keep this from continuing. Thanks, -Matt ____________________________________ Matthew R. Norman Computational Climate Scientist Scientific Computing Group National Center for Computational Sciences Oak Ridge National Laboratory
I think this is a machine dependent default in CIME. So Min could submit a PR to make this the default? On Fri, Feb 9, 2018 at 1:44 PM, Norman, Matthew R. <[email protected]> wrote:
Hi all,
There have been some problems with "./create_test" on Titan eating up too many resources on the external login nodes, and admins have had to kill some of the jobs. I assume this is because of the build phase, so for now, please run ./create_test with "-j 1" (see ./create_test --help) when running on Titan to keep this from continuing. Thanks,
-Matt
____________________________________ Matthew R. Norman Computational Climate Scientist Scientific Computing Group National Center for Computational Sciences Oak Ridge National Laboratory
-- You received this message because you are subscribed to the Google Groups "acme-ecp" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/ msgid/acme-ecp/1518209083999.73585%40ornl.gov <https://groups.google.com/d/msgid/acme-ecp/1518209083999.73585%40ornl.gov?utm_medium=email&utm_source=footer> . For more options, visit https://groups.google.com/d/optout.
Mark, I wonder if it was Wade's testing that caused the problem? There are many ways to throttle the resources used by create_test. You can use "create_test -j N" to reduce the task-level parallelism of create_test (how many test cases it will process simultaneously). You can use "create_test --proc-pool=P" to reduce the number of processors create_test will use. Finally, you can reduce the parallelism of individual case builds by changing the GMAKE_J value in config_machines.xml. If Wade is the cause of the problem, then the correct approach is probably to throttle the task-level parallelism. -Jim From: ACME-SE <[email protected]> on behalf of Mark Taylor <[email protected]> Date: Sunday, February 11, 2018 at 12:37 PM To: "Norman, Matthew R." <[email protected]> Cc: "Vergara Larrea, Veronica G." <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]>, "[email protected]" <[email protected]> Subject: [EXTERNAL] Re: [ACME-SE] Please ./create_test with "-j 1" on Titan I think this is a machine dependent default in CIME. So Min could submit a PR to make this the default? On Fri, Feb 9, 2018 at 1:44 PM, Norman, Matthew R. <[email protected]<mailto:[email protected]>> wrote: Hi all, There have been some problems with "./create_test" on Titan eating up too many resources on the external login nodes, and admins have had to kill some of the jobs. I assume this is because of the build phase, so for now, please run ./create_test with "-j 1" (see ./create_test --help) when running on Titan to keep this from continuing. Thanks, -Matt ____________________________________ Matthew R. Norman Computational Climate Scientist Scientific Computing Group National Center for Computational Sciences Oak Ridge National Laboratory -- You received this message because you are subscribed to the Google Groups "acme-ecp" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]<mailto:[email protected]>. To post to this group, send email to [email protected]<mailto:[email protected]>. To view this discussion on the web visit https://groups.google.com/d/msgid/acme-ecp/1518209083999.73585%40ornl.gov<https://groups.google.com/d/msgid/acme-ecp/1518209083999.73585%40ornl.gov?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout.
participants (3)
-
Foucar, James G -
Mark Taylor -
Norman, Matthew R.