December 2014 - swift-bugs - mailman3.cels.anl.gov

[Bug 1141] New: Null pointer exception if foreach array is empty
by bugzilla-daemon＠mcs.anl.gov 30 Jul '15

30 Jul '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1141 Summary: Null pointer exception if foreach array is empty Product: Swift Version: 0.94 Platform: PC OS/Version: Mac OS Status: ASSIGNED Severity: normal Priority: P2 Component: SwiftScript language AssignedTo: hategan(a)mcs.anl.gov ReportedBy: wilde(a)mcs.anl.gov Example below. Note that the null array is desired here, as the array may be set dynamically and an empty array should result in zero iterations of the foreach. login1$ swift e1.swift WARNING: Empty array constant Swift 0.94.1 swift-r7114 cog-r3803 RunID: 20131123-1240-v2j459gd Progress: time: Sat, 23 Nov 2013 12:40:31 -0600 Execution failed: java.lang.NullPointerException Caused by: java.lang.NullPointerException at org.griphyn.vdl.karajan.lib.CreateArray.function(CreateArray.java:57) at org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:81) at org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.completed(AbstractSequentialWithArguments.java:194) at org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:214) at org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58) at org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:28) at org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:29) at org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:20) at org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63) at org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:139) at org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:197) at org.globus.cog.karajan.workflow.FlowElementWrapper.start(FlowElementWrapper.java:227) at org.globus.cog.karajan.workflow.events.EventBus.start(EventBus.java:103) at org.globus.cog.karajan.workflow.events.EventTargetPair.run(EventTargetPair.java:40) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) login1$ cat e1.swift int i[] = [1,3,5,7,9]; foreach j in [] { trace(j); } login1$ -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 945] New: coaster.service.job.manager.SortedJobSet NoSuchElementException
by bugzilla-daemon＠mcs.anl.gov 30 Jul '15

30 Jul '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=945 Summary: coaster.service.job.manager.SortedJobSet NoSuchElementException Product: Swift Version: trunk Platform: PC OS/Version: Mac OS Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: davidk(a)ci.uchicago.edu I am seeing this error on Beagle: CoasterService fatal error: Exception caught in block processor java.util.NoSuchElementException at org.globus.cog.abstraction.coaster.service.job.manager.SortedJobSet$1.next(SortedJobSet.java:182) at org.globus.cog.abstraction.coaster.service.job.manager.SortedJobSet$1.next(SortedJobSet.java:162) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.allocateBlocks(BlockQueueProcessor.java:609) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.updatePlan(BlockQueueProcessor.java:734) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.run(BlockQueueProcessor.java:139) Exception caught in block processor java.util.NoSuchElementException at org.globus.cog.abstraction.coaster.service.job.manager.SortedJobSet$1.next(SortedJobSet.java:182) at org.globus.cog.abstraction.coaster.service.job.manager.SortedJobSet$1.next(SortedJobSet.java:162) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.allocateBlocks(BlockQueueProcessor.java:609) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.updatePlan(BlockQueueProcessor.java:734) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.run(BlockQueueProcessor.java:139) The full log is at http://www.ci.uchicago.edu/~davidk/logs/modis-20130130-1819-jrybzslc.log -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 9

[Bug 714] New: Give more information for "could not convert" error message
by bugzilla-daemon＠mcs.anl.gov 19 Jun '15

19 Jun '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=714 Summary: Give more information for "could not convert" error message Product: Swift Version: 0.93 Platform: PC OS/Version: Mac OS Status: NEW Severity: normal Priority: P2 Component: SwiftScript language AssignedTo: davidk(a)ci.uchicago.edu ReportedBy: wilde(a)mcs.anl.gov Swift dies with the following message: Execution failed: Could not convert value to number: nan Provide at least the built-in function name and ideally the line number of the failing operation. -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

[Bug 710] New: Keyword argument passing doesnt seem to work as described in user guide
by bugzilla-daemon＠mcs.anl.gov 06 Mar '15

06 Mar '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=710 Summary: Keyword argument passing doesnt seem to work as described in user guide Product: Swift Version: 0.93 Platform: PC OS/Version: Mac OS Status: NEW Severity: minor Priority: P4 Component: SwiftScript language AssignedTo: hategan(a)mcs.anl.gov ReportedBy: wilde(a)mcs.anl.gov (need to provide test cases) -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

[Bug 1367] New: Multi-site configs run extremely slow.
by bugzilla-daemon＠mcs.anl.gov 15 Feb '15

15 Feb '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1367 Summary: Multi-site configs run extremely slow. Product: Swift Version: 0.96 Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: yadunandb(a)ci.uchicago.edu The task defined is echo "hi" with the app taking no input or output files. Each swift run calls the app 1000 times in a foreach loop. With time swift -sites crush task_pump.swift -loops=1000 -> real 0m27.436s With time swift -sites crank task_pump.swift -loops=1000 -> real 0m19.343s With time swift -sites crush,crank task_pump.swift -loops=1000 -> real 5m12.178s It looks like the run stalls in the beginning and at the end, here's a paste of the swift.out : Swift trunk git-rev: b72672ca81a5accfa7c2857e09f0314f80c7681a heads/master 6145 (modified locally) RunID: run004 Progress: Thu, 23 Oct 2014 19:16:28+0000 Progress: Thu, 23 Oct 2014 19:16:29+0000 Selecting site:983 Submitting:18 Progress: Thu, 23 Oct 2014 19:16:35+0000 Selecting site:981 Submitting:18 Finished successfully:2 Progress: Thu, 23 Oct 2014 19:17:05+0000 Selecting site:981 Submitting:18 Finished successfully:2 Progress: Thu, 23 Oct 2014 19:17:08+0000 Selecting site:981 Submitted:18 Finished successfully:2 Progress: Thu, 23 Oct 2014 19:17:11+0000 Selecting site:981 Stage in:13 Submitted:1 Active:4 Finished successfully:2 Progress: Thu, 23 Oct 2014 19:17:18+0000 Selecting site:885 Stage in:2 Submitting:3 Submitted:3 Stage out:10 Finished successfully:98 Progress: Thu, 23 Oct 2014 19:17:19+0000 Selecting site:711 Stage in:9 Submitting:4 Submitted:3 Active:1 Stage out:1 Finished successfully:272 Progress: Thu, 23 Oct 2014 19:17:20+0000 Selecting site:578 Stage in:1 Submitted:6 Active:7 Stage out:3 Finished successfully:406 Progress: Thu, 23 Oct 2014 19:17:21+0000 Selecting site:416 Stage in:2 Submitting:4 Active:6 Stage out:6 Finished successfully:567 Progress: Thu, 23 Oct 2014 19:17:22+0000 Selecting site:206 Stage in:6 Submitting:8 Active:2 Stage out:2 Finished successfully:777 Progress: Thu, 23 Oct 2014 19:17:23+0000 Selecting site:12 Stage in:3 Submitting:9 Active:2 Stage out:4 Finished successfully:971 Progress: Thu, 23 Oct 2014 19:17:24+0000 Active:2 Stage out:1 Finished successfully:998 Progress: Thu, 23 Oct 2014 19:17:54+0000 Active:2 Stage out:1 Finished successfully:998 Progress: Thu, 23 Oct 2014 19:17:55+0000 Active:1 Stage out:1 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:18:25+0000 Active:1 Stage out:1 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:18:33+0000 Stage out:2 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:19:03+0000 Stage out:2 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:19:33+0000 Stage out:2 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:20:03+0000 Stage out:2 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:20:33+0000 Stage out:2 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:21:03+0000 Stage out:2 Finished successfully:999 Final status:Thu, 23 Oct 2014 19:21:28+0000 Finished successfully:1001 Here are run directories from the three runs : http://users.rcc.uchicago.edu/~yadunand/task_pump/run001/ http://users.rcc.uchicago.edu/~yadunand/task_pump/run002/ http://users.rcc.uchicago.edu/~yadunand/task_pump/run004/ -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 8

[Bug 1371] New: Aprun not working on beagle with trunk
by bugzilla-daemon＠mcs.anl.gov 15 Jan '15

15 Jan '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1371 Summary: Aprun not working on beagle with trunk Product: Swift Version: 0.96 Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: yadunandb(a)ci.uchicago.edu Copied from Bug 1325. These tests were run on Beagle from midway with ssh-cl:pbs. With no jobOptions to pbs specified, the swift run completes successfully. With the following config options specified the swift run completes successfully: jobOptions { pbs.mpp: true depth: "24" } In the two cases above, the test scripts show that the scripts were executed on the mom nodes and not the compute nodes, which isn't the behavior we want on beagle. With the following configs specified the swift run fails. jobOptions { pbs.aprun: true pbs.mpp: true depth: "24" } Here's the run folders from all three cases : http://users.rcc.uchicago.edu/~yadunand/beagle_aprun_fail.tar.gz Here's the coaster log from Beagle : http://users.rcc.uchicago.edu/~yadunand/coasters-aprun-bug.log Another coaster log from beagle with just the failure with aprun config http://users.rcc.uchicago.edu/~yadunand/coasters-only-aprun.log The beagle_sanity test at swift-k/tests/sites/beagle should reproduce this behavior. (Tested with Swift trunk git-rev: b72672ca81a5accfa7c2857e09f0314f80c7681a heads/master 6145) -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

[Bug 1381] New: Requesting an updated 0.95 on Midway
by bugzilla-daemon＠mcs.anl.gov 30 Dec '14

30 Dec '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1381 Summary: Requesting an updated 0.95 on Midway Product: Swift Version: 0.95 Platform: PC OS/Version: Mac OS Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: yadudoc1729(a)gmail.com ReportedBy: davidkelly(a)uchicago.edu Hello, The existing 0.95 module on midway (swift/0.95-RC5) has an issue where it prints a ton of java errors at the end of large runs. This bug (https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1265) has been fixed, but a fixed version is not available as a module on Midway. Could we please make the latest 0.95 available as a module on Midway? Thanks, David -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 1387] New: Unable to specify sge parallel environment [trunk]
by bugzilla-daemon＠mcs.anl.gov 30 Dec '14

30 Dec '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1387 Summary: Unable to specify sge parallel environment [trunk] Product: Swift Version: 0.96 Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: yadunandb(a)ci.uchicago.edu Please document how to set parallel.environment (pe) for sge systems. It is also unclear in the documentation whether coasters works for the sge provider. Here's the config I've got so far : site.godzilla { execution { type: "coaster" jobManager: "local:sge" options { nodeGranularity: 1 maxNodesPerJob: 1 jobQueue: "all.q" maxJobs: 1 tasksPerNode: 1 maxJobTime: "00:08:20" # "pe": "make" } } staging: local workDirectory: "/tmp/"${env.USER} app.date { executable: "/bin/date" maxWallTime: "00:05:00" } } sites: godzilla This fails with the error : Could not submit job (qsub reported an exit code of 1). Unable to run job: job rejected: the requested parallel environment "1way" does not exist.Exiting. I updated the etc/provider-sge.properties with parallel.environment=make and that seems to have generated a config which the scheduler does not reject immediately. I do not know what config option in the swift.conf lets you set this property. -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 1391] New: app definitions working incorrectly on trunk
by bugzilla-daemon＠mcs.anl.gov 29 Dec '14

29 Dec '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1391 Summary: app definitions working incorrectly on trunk Product: Swift Version: 0.96 Platform: PC OS/Version: Linux Status: NEW Severity: critical Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: yadunandb(a)ci.uchicago.edu The swift remote test driver does this : app (...) mid_run (...){ mid @scripts... } app (...) osg_run (...){ osg @scripts... } mid_run(testrunner.sh) osg_run(testrunner.sh) The swift.conf defines mid for the midway site as bash, and osg for osgconnect as bash. This seems to have broken between Dec 4 and now. I added some cleanup codes to remove test results older than 5 days, because the midway folks reduced disk space. So, I'm having trouble figuring out when this might have broken. This is the error I get on stdout: Exception in mid: Arguments: [wrapper.sh, swift.tar, out/Logs_0002.tar] Host: local Directory: remote_driver-run001/jobs/m/mid-yd54yr1m exception @ swift-int-staging.k, line: 165 Caused by: Cannot find executable mid on site system path exception @ swift-int-staging.k, line: 160 Caused by: Process completed with exit code 254 This is blocking the test-suite. Here's the test suite failures : http://users.rcc.uchicago.edu/~yadunand/swift-k/run-2014-12-22/remote_drive… -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

[Bug 1369] New: NPE with filesys_mapper
by bugzilla-daemon＠mcs.anl.gov 27 Dec '14

27 Dec '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1369 Summary: NPE with filesys_mapper Product: Swift Version: 0.96 Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: yadunandb(a)ci.uchicago.edu The following piece of code fails with a NullPointerException, when the file arrays out and err are using filesys_mapper. Changing the mapper to simple_mapper, gets the code running without issues. type file; int loops = toInt(arg("loops","50")); int delay = toInt(arg("delay", "1")); app (file out, file err) app_foo (file script, int delay) { bash @script delay stdout=@out stderr=@err; } file out[] <filesys_mapper; location="output", prefix="std", suffix=".out">; file err[] <filesys_mapper; location="output", prefix="std", suffix=".err">; //file out[] <simple_mapper; location="output", prefix="foo.", suffix=".out">; //file err[] <simple_mapper; location="output", prefix="foo.", suffix=".err">; file wrapper <"wrapper.sh">; foreach i in [1:loops] { tracef("Count : %i \n", i); (out[i], err[i]) = app_foo (wrapper, delay); } The logs from the run are here : http://users.rcc.uchicago.edu/~yadunand/filesys_mapper_NPE/ -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3