November 2014 - swift-bugs - mailman3.cels.anl.gov

[Bug 1141] New: Null pointer exception if foreach array is empty
by bugzilla-daemon＠mcs.anl.gov 30 Jul '15

30 Jul '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1141 Summary: Null pointer exception if foreach array is empty Product: Swift Version: 0.94 Platform: PC OS/Version: Mac OS Status: ASSIGNED Severity: normal Priority: P2 Component: SwiftScript language AssignedTo: hategan(a)mcs.anl.gov ReportedBy: wilde(a)mcs.anl.gov Example below. Note that the null array is desired here, as the array may be set dynamically and an empty array should result in zero iterations of the foreach. login1$ swift e1.swift WARNING: Empty array constant Swift 0.94.1 swift-r7114 cog-r3803 RunID: 20131123-1240-v2j459gd Progress: time: Sat, 23 Nov 2013 12:40:31 -0600 Execution failed: java.lang.NullPointerException Caused by: java.lang.NullPointerException at org.griphyn.vdl.karajan.lib.CreateArray.function(CreateArray.java:57) at org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:81) at org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.completed(AbstractSequentialWithArguments.java:194) at org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:214) at org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58) at org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:28) at org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:29) at org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:20) at org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63) at org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:139) at org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:197) at org.globus.cog.karajan.workflow.FlowElementWrapper.start(FlowElementWrapper.java:227) at org.globus.cog.karajan.workflow.events.EventBus.start(EventBus.java:103) at org.globus.cog.karajan.workflow.events.EventTargetPair.run(EventTargetPair.java:40) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) login1$ cat e1.swift int i[] = [1,3,5,7,9]; foreach j in [] { trace(j); } login1$ -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 945] New: coaster.service.job.manager.SortedJobSet NoSuchElementException
by bugzilla-daemon＠mcs.anl.gov 30 Jul '15

30 Jul '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=945 Summary: coaster.service.job.manager.SortedJobSet NoSuchElementException Product: Swift Version: trunk Platform: PC OS/Version: Mac OS Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: davidk(a)ci.uchicago.edu I am seeing this error on Beagle: CoasterService fatal error: Exception caught in block processor java.util.NoSuchElementException at org.globus.cog.abstraction.coaster.service.job.manager.SortedJobSet$1.next(SortedJobSet.java:182) at org.globus.cog.abstraction.coaster.service.job.manager.SortedJobSet$1.next(SortedJobSet.java:162) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.allocateBlocks(BlockQueueProcessor.java:609) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.updatePlan(BlockQueueProcessor.java:734) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.run(BlockQueueProcessor.java:139) Exception caught in block processor java.util.NoSuchElementException at org.globus.cog.abstraction.coaster.service.job.manager.SortedJobSet$1.next(SortedJobSet.java:182) at org.globus.cog.abstraction.coaster.service.job.manager.SortedJobSet$1.next(SortedJobSet.java:162) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.allocateBlocks(BlockQueueProcessor.java:609) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.updatePlan(BlockQueueProcessor.java:734) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.run(BlockQueueProcessor.java:139) The full log is at http://www.ci.uchicago.edu/~davidk/logs/modis-20130130-1819-jrybzslc.log -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 9

[Bug 714] New: Give more information for "could not convert" error message
by bugzilla-daemon＠mcs.anl.gov 19 Jun '15

19 Jun '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=714 Summary: Give more information for "could not convert" error message Product: Swift Version: 0.93 Platform: PC OS/Version: Mac OS Status: NEW Severity: normal Priority: P2 Component: SwiftScript language AssignedTo: davidk(a)ci.uchicago.edu ReportedBy: wilde(a)mcs.anl.gov Swift dies with the following message: Execution failed: Could not convert value to number: nan Provide at least the built-in function name and ideally the line number of the failing operation. -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

[Bug 710] New: Keyword argument passing doesnt seem to work as described in user guide
by bugzilla-daemon＠mcs.anl.gov 06 Mar '15

06 Mar '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=710 Summary: Keyword argument passing doesnt seem to work as described in user guide Product: Swift Version: 0.93 Platform: PC OS/Version: Mac OS Status: NEW Severity: minor Priority: P4 Component: SwiftScript language AssignedTo: hategan(a)mcs.anl.gov ReportedBy: wilde(a)mcs.anl.gov (need to provide test cases) -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

[Bug 1367] New: Multi-site configs run extremely slow.
by bugzilla-daemon＠mcs.anl.gov 15 Feb '15

15 Feb '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1367 Summary: Multi-site configs run extremely slow. Product: Swift Version: 0.96 Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: yadunandb(a)ci.uchicago.edu The task defined is echo "hi" with the app taking no input or output files. Each swift run calls the app 1000 times in a foreach loop. With time swift -sites crush task_pump.swift -loops=1000 -> real 0m27.436s With time swift -sites crank task_pump.swift -loops=1000 -> real 0m19.343s With time swift -sites crush,crank task_pump.swift -loops=1000 -> real 5m12.178s It looks like the run stalls in the beginning and at the end, here's a paste of the swift.out : Swift trunk git-rev: b72672ca81a5accfa7c2857e09f0314f80c7681a heads/master 6145 (modified locally) RunID: run004 Progress: Thu, 23 Oct 2014 19:16:28+0000 Progress: Thu, 23 Oct 2014 19:16:29+0000 Selecting site:983 Submitting:18 Progress: Thu, 23 Oct 2014 19:16:35+0000 Selecting site:981 Submitting:18 Finished successfully:2 Progress: Thu, 23 Oct 2014 19:17:05+0000 Selecting site:981 Submitting:18 Finished successfully:2 Progress: Thu, 23 Oct 2014 19:17:08+0000 Selecting site:981 Submitted:18 Finished successfully:2 Progress: Thu, 23 Oct 2014 19:17:11+0000 Selecting site:981 Stage in:13 Submitted:1 Active:4 Finished successfully:2 Progress: Thu, 23 Oct 2014 19:17:18+0000 Selecting site:885 Stage in:2 Submitting:3 Submitted:3 Stage out:10 Finished successfully:98 Progress: Thu, 23 Oct 2014 19:17:19+0000 Selecting site:711 Stage in:9 Submitting:4 Submitted:3 Active:1 Stage out:1 Finished successfully:272 Progress: Thu, 23 Oct 2014 19:17:20+0000 Selecting site:578 Stage in:1 Submitted:6 Active:7 Stage out:3 Finished successfully:406 Progress: Thu, 23 Oct 2014 19:17:21+0000 Selecting site:416 Stage in:2 Submitting:4 Active:6 Stage out:6 Finished successfully:567 Progress: Thu, 23 Oct 2014 19:17:22+0000 Selecting site:206 Stage in:6 Submitting:8 Active:2 Stage out:2 Finished successfully:777 Progress: Thu, 23 Oct 2014 19:17:23+0000 Selecting site:12 Stage in:3 Submitting:9 Active:2 Stage out:4 Finished successfully:971 Progress: Thu, 23 Oct 2014 19:17:24+0000 Active:2 Stage out:1 Finished successfully:998 Progress: Thu, 23 Oct 2014 19:17:54+0000 Active:2 Stage out:1 Finished successfully:998 Progress: Thu, 23 Oct 2014 19:17:55+0000 Active:1 Stage out:1 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:18:25+0000 Active:1 Stage out:1 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:18:33+0000 Stage out:2 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:19:03+0000 Stage out:2 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:19:33+0000 Stage out:2 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:20:03+0000 Stage out:2 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:20:33+0000 Stage out:2 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:21:03+0000 Stage out:2 Finished successfully:999 Final status:Thu, 23 Oct 2014 19:21:28+0000 Finished successfully:1001 Here are run directories from the three runs : http://users.rcc.uchicago.edu/~yadunand/task_pump/run001/ http://users.rcc.uchicago.edu/~yadunand/task_pump/run002/ http://users.rcc.uchicago.edu/~yadunand/task_pump/run004/ -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 8

[Bug 1371] New: Aprun not working on beagle with trunk
by bugzilla-daemon＠mcs.anl.gov 15 Jan '15

15 Jan '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1371 Summary: Aprun not working on beagle with trunk Product: Swift Version: 0.96 Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: yadunandb(a)ci.uchicago.edu Copied from Bug 1325. These tests were run on Beagle from midway with ssh-cl:pbs. With no jobOptions to pbs specified, the swift run completes successfully. With the following config options specified the swift run completes successfully: jobOptions { pbs.mpp: true depth: "24" } In the two cases above, the test scripts show that the scripts were executed on the mom nodes and not the compute nodes, which isn't the behavior we want on beagle. With the following configs specified the swift run fails. jobOptions { pbs.aprun: true pbs.mpp: true depth: "24" } Here's the run folders from all three cases : http://users.rcc.uchicago.edu/~yadunand/beagle_aprun_fail.tar.gz Here's the coaster log from Beagle : http://users.rcc.uchicago.edu/~yadunand/coasters-aprun-bug.log Another coaster log from beagle with just the failure with aprun config http://users.rcc.uchicago.edu/~yadunand/coasters-only-aprun.log The beagle_sanity test at swift-k/tests/sites/beagle should reproduce this behavior. (Tested with Swift trunk git-rev: b72672ca81a5accfa7c2857e09f0314f80c7681a heads/master 6145) -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

[Bug 1369] New: NPE with filesys_mapper
by bugzilla-daemon＠mcs.anl.gov 27 Dec '14

27 Dec '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1369 Summary: NPE with filesys_mapper Product: Swift Version: 0.96 Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: yadunandb(a)ci.uchicago.edu The following piece of code fails with a NullPointerException, when the file arrays out and err are using filesys_mapper. Changing the mapper to simple_mapper, gets the code running without issues. type file; int loops = toInt(arg("loops","50")); int delay = toInt(arg("delay", "1")); app (file out, file err) app_foo (file script, int delay) { bash @script delay stdout=@out stderr=@err; } file out[] <filesys_mapper; location="output", prefix="std", suffix=".out">; file err[] <filesys_mapper; location="output", prefix="std", suffix=".err">; //file out[] <simple_mapper; location="output", prefix="foo.", suffix=".out">; //file err[] <simple_mapper; location="output", prefix="foo.", suffix=".err">; file wrapper <"wrapper.sh">; foreach i in [1:loops] { tracef("Count : %i \n", i); (out[i], err[i]) = app_foo (wrapper, delay); } The logs from the run are here : http://users.rcc.uchicago.edu/~yadunand/filesys_mapper_NPE/ -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

[Bug 1377] New: Connection to workers lost/Channel timeout on OSG
by bugzilla-daemon＠mcs.anl.gov 20 Nov '14

20 Nov '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1377 Summary: Connection to workers lost/Channel timeout on OSG Product: Swift Version: 0.96 Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: yadunandb(a)ci.uchicago.edu For long running jobs (workers are up for ~2hrs), I'm seeing several workers fail with channel timeouts, and the following error message in the swift.log: 2014-11-10 11:30:41,367-0600 DEBUG swift APPLICATION_EXCEPTION jobid=bash-1ip8c30m - Application exception: exception @ swift-int-staging.k, line: 160 Caused by: Block task failed: Connection to worker lost java.io.EOFException at org.globus.cog.coaster.channels.AbstractStreamCoasterChannel.readFromChannel(AbstractStreamCoasterChannel.java:253) at org.globus.cog.coaster.channels.AbstractStreamCoasterChannel.stepNIO(AbstractStreamCoasterChannel.java:186) at org.globus.cog.coaster.channels.NIOMultiplexer.loop(NIOMultiplexer.java:116) at org.globus.cog.coaster.channels.NIOMultiplexer.run(NIOMultiplexer.java:75) Of the 956 Condor*submit scripts, I see 31 end with Channeltimeouts in the stderr logs. The concern is that due to these worker failures, even with lazyErrors=true and executionRetries set to 2, the application ends up failing. I have not dug deep enough to understand if there is a bug in the worker code that is resulting in the failures, or if this is a reasonable level of reliability from OSG. (31/956 =~ 3%failure). Here are the logs : http://stash.osgconnect.net/+yadunand/run033.tar.gz http://stash.osgconnect.net/+yadunand/worker-logs.tar.gz -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

[Bug 1375] New: Direct staging documentation
by bugzilla-daemon＠mcs.anl.gov 11 Nov '14

11 Nov '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1375 Summary: Direct staging documentation Product: Swift Version: 0.96 Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: P2 Component: Documentation AssignedTo: hategan(a)mcs.anl.gov ReportedBy: yadunandb(a)ci.uchicago.edu The trunk documentation does not contain information on direct staging. It would also help to note additional variants that can be used along with direct staging like adding a workernode-local scratch disk. -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 0

[Bug 1373] New: Suppress no-script message for swift -version
by bugzilla-daemon＠mcs.anl.gov 08 Nov '14

08 Nov '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1373 Summary: Suppress no-script message for swift -version Product: Swift Version: 0.96 Platform: PC OS/Version: Mac OS Status: ASSIGNED Severity: trivial Priority: P2 Component: Error messages AssignedTo: yadunandb(a)ci.uchicago.edu ReportedBy: wilde(a)mcs.anl.gov CC: wozniak(a)mcs.anl.gov swift$ swift -v Swift trunk git-rev: 7a0c4927380514e4c7166dfd095fd0a4c6389b03 heads/master 6172 No Swift script specified swift$ The "No Swift script specified" should be supressed when the says just "-v" or "-version". -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1