October 2014 - swift-bugs - mailman3.cels.anl.gov

[Bug 1141] New: Null pointer exception if foreach array is empty
by bugzilla-daemon＠mcs.anl.gov 30 Jul '15

30 Jul '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1141 Summary: Null pointer exception if foreach array is empty Product: Swift Version: 0.94 Platform: PC OS/Version: Mac OS Status: ASSIGNED Severity: normal Priority: P2 Component: SwiftScript language AssignedTo: hategan(a)mcs.anl.gov ReportedBy: wilde(a)mcs.anl.gov Example below. Note that the null array is desired here, as the array may be set dynamically and an empty array should result in zero iterations of the foreach. login1$ swift e1.swift WARNING: Empty array constant Swift 0.94.1 swift-r7114 cog-r3803 RunID: 20131123-1240-v2j459gd Progress: time: Sat, 23 Nov 2013 12:40:31 -0600 Execution failed: java.lang.NullPointerException Caused by: java.lang.NullPointerException at org.griphyn.vdl.karajan.lib.CreateArray.function(CreateArray.java:57) at org.griphyn.vdl.karajan.lib.VDLFunction.post(VDLFunction.java:81) at org.globus.cog.karajan.workflow.nodes.AbstractSequentialWithArguments.completed(AbstractSequentialWithArguments.java:194) at org.globus.cog.karajan.workflow.nodes.FlowNode.complete(FlowNode.java:214) at org.globus.cog.karajan.workflow.nodes.FlowContainer.post(FlowContainer.java:58) at org.globus.cog.karajan.workflow.nodes.functions.AbstractFunction.post(AbstractFunction.java:28) at org.globus.cog.karajan.workflow.nodes.Sequential.startNext(Sequential.java:29) at org.globus.cog.karajan.workflow.nodes.Sequential.executeChildren(Sequential.java:20) at org.globus.cog.karajan.workflow.nodes.FlowContainer.execute(FlowContainer.java:63) at org.globus.cog.karajan.workflow.nodes.FlowNode.restart(FlowNode.java:139) at org.globus.cog.karajan.workflow.nodes.FlowNode.start(FlowNode.java:197) at org.globus.cog.karajan.workflow.FlowElementWrapper.start(FlowElementWrapper.java:227) at org.globus.cog.karajan.workflow.events.EventBus.start(EventBus.java:103) at org.globus.cog.karajan.workflow.events.EventTargetPair.run(EventTargetPair.java:40) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:722) login1$ cat e1.swift int i[] = [1,3,5,7,9]; foreach j in [] { trace(j); } login1$ -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 945] New: coaster.service.job.manager.SortedJobSet NoSuchElementException
by bugzilla-daemon＠mcs.anl.gov 30 Jul '15

30 Jul '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=945 Summary: coaster.service.job.manager.SortedJobSet NoSuchElementException Product: Swift Version: trunk Platform: PC OS/Version: Mac OS Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: davidk(a)ci.uchicago.edu I am seeing this error on Beagle: CoasterService fatal error: Exception caught in block processor java.util.NoSuchElementException at org.globus.cog.abstraction.coaster.service.job.manager.SortedJobSet$1.next(SortedJobSet.java:182) at org.globus.cog.abstraction.coaster.service.job.manager.SortedJobSet$1.next(SortedJobSet.java:162) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.allocateBlocks(BlockQueueProcessor.java:609) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.updatePlan(BlockQueueProcessor.java:734) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.run(BlockQueueProcessor.java:139) Exception caught in block processor java.util.NoSuchElementException at org.globus.cog.abstraction.coaster.service.job.manager.SortedJobSet$1.next(SortedJobSet.java:182) at org.globus.cog.abstraction.coaster.service.job.manager.SortedJobSet$1.next(SortedJobSet.java:162) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.allocateBlocks(BlockQueueProcessor.java:609) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.updatePlan(BlockQueueProcessor.java:734) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.run(BlockQueueProcessor.java:139) The full log is at http://www.ci.uchicago.edu/~davidk/logs/modis-20130130-1819-jrybzslc.log -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 9

[Bug 714] New: Give more information for "could not convert" error message
by bugzilla-daemon＠mcs.anl.gov 19 Jun '15

19 Jun '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=714 Summary: Give more information for "could not convert" error message Product: Swift Version: 0.93 Platform: PC OS/Version: Mac OS Status: NEW Severity: normal Priority: P2 Component: SwiftScript language AssignedTo: davidk(a)ci.uchicago.edu ReportedBy: wilde(a)mcs.anl.gov Swift dies with the following message: Execution failed: Could not convert value to number: nan Provide at least the built-in function name and ideally the line number of the failing operation. -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

[Bug 710] New: Keyword argument passing doesnt seem to work as described in user guide
by bugzilla-daemon＠mcs.anl.gov 06 Mar '15

06 Mar '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=710 Summary: Keyword argument passing doesnt seem to work as described in user guide Product: Swift Version: 0.93 Platform: PC OS/Version: Mac OS Status: NEW Severity: minor Priority: P4 Component: SwiftScript language AssignedTo: hategan(a)mcs.anl.gov ReportedBy: wilde(a)mcs.anl.gov (need to provide test cases) -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

[Bug 1367] New: Multi-site configs run extremely slow.
by bugzilla-daemon＠mcs.anl.gov 15 Feb '15

15 Feb '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1367 Summary: Multi-site configs run extremely slow. Product: Swift Version: 0.96 Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: yadunandb(a)ci.uchicago.edu The task defined is echo "hi" with the app taking no input or output files. Each swift run calls the app 1000 times in a foreach loop. With time swift -sites crush task_pump.swift -loops=1000 -> real 0m27.436s With time swift -sites crank task_pump.swift -loops=1000 -> real 0m19.343s With time swift -sites crush,crank task_pump.swift -loops=1000 -> real 5m12.178s It looks like the run stalls in the beginning and at the end, here's a paste of the swift.out : Swift trunk git-rev: b72672ca81a5accfa7c2857e09f0314f80c7681a heads/master 6145 (modified locally) RunID: run004 Progress: Thu, 23 Oct 2014 19:16:28+0000 Progress: Thu, 23 Oct 2014 19:16:29+0000 Selecting site:983 Submitting:18 Progress: Thu, 23 Oct 2014 19:16:35+0000 Selecting site:981 Submitting:18 Finished successfully:2 Progress: Thu, 23 Oct 2014 19:17:05+0000 Selecting site:981 Submitting:18 Finished successfully:2 Progress: Thu, 23 Oct 2014 19:17:08+0000 Selecting site:981 Submitted:18 Finished successfully:2 Progress: Thu, 23 Oct 2014 19:17:11+0000 Selecting site:981 Stage in:13 Submitted:1 Active:4 Finished successfully:2 Progress: Thu, 23 Oct 2014 19:17:18+0000 Selecting site:885 Stage in:2 Submitting:3 Submitted:3 Stage out:10 Finished successfully:98 Progress: Thu, 23 Oct 2014 19:17:19+0000 Selecting site:711 Stage in:9 Submitting:4 Submitted:3 Active:1 Stage out:1 Finished successfully:272 Progress: Thu, 23 Oct 2014 19:17:20+0000 Selecting site:578 Stage in:1 Submitted:6 Active:7 Stage out:3 Finished successfully:406 Progress: Thu, 23 Oct 2014 19:17:21+0000 Selecting site:416 Stage in:2 Submitting:4 Active:6 Stage out:6 Finished successfully:567 Progress: Thu, 23 Oct 2014 19:17:22+0000 Selecting site:206 Stage in:6 Submitting:8 Active:2 Stage out:2 Finished successfully:777 Progress: Thu, 23 Oct 2014 19:17:23+0000 Selecting site:12 Stage in:3 Submitting:9 Active:2 Stage out:4 Finished successfully:971 Progress: Thu, 23 Oct 2014 19:17:24+0000 Active:2 Stage out:1 Finished successfully:998 Progress: Thu, 23 Oct 2014 19:17:54+0000 Active:2 Stage out:1 Finished successfully:998 Progress: Thu, 23 Oct 2014 19:17:55+0000 Active:1 Stage out:1 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:18:25+0000 Active:1 Stage out:1 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:18:33+0000 Stage out:2 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:19:03+0000 Stage out:2 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:19:33+0000 Stage out:2 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:20:03+0000 Stage out:2 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:20:33+0000 Stage out:2 Finished successfully:999 Progress: Thu, 23 Oct 2014 19:21:03+0000 Stage out:2 Finished successfully:999 Final status:Thu, 23 Oct 2014 19:21:28+0000 Finished successfully:1001 Here are run directories from the three runs : http://users.rcc.uchicago.edu/~yadunand/task_pump/run001/ http://users.rcc.uchicago.edu/~yadunand/task_pump/run002/ http://users.rcc.uchicago.edu/~yadunand/task_pump/run004/ -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 8

[Bug 1371] New: Aprun not working on beagle with trunk
by bugzilla-daemon＠mcs.anl.gov 15 Jan '15

15 Jan '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1371 Summary: Aprun not working on beagle with trunk Product: Swift Version: 0.96 Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: yadunandb(a)ci.uchicago.edu Copied from Bug 1325. These tests were run on Beagle from midway with ssh-cl:pbs. With no jobOptions to pbs specified, the swift run completes successfully. With the following config options specified the swift run completes successfully: jobOptions { pbs.mpp: true depth: "24" } In the two cases above, the test scripts show that the scripts were executed on the mom nodes and not the compute nodes, which isn't the behavior we want on beagle. With the following configs specified the swift run fails. jobOptions { pbs.aprun: true pbs.mpp: true depth: "24" } Here's the run folders from all three cases : http://users.rcc.uchicago.edu/~yadunand/beagle_aprun_fail.tar.gz Here's the coaster log from Beagle : http://users.rcc.uchicago.edu/~yadunand/coasters-aprun-bug.log Another coaster log from beagle with just the failure with aprun config http://users.rcc.uchicago.edu/~yadunand/coasters-only-aprun.log The beagle_sanity test at swift-k/tests/sites/beagle should reproduce this behavior. (Tested with Swift trunk git-rev: b72672ca81a5accfa7c2857e09f0314f80c7681a heads/master 6145) -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

[Bug 1369] New: NPE with filesys_mapper
by bugzilla-daemon＠mcs.anl.gov 27 Dec '14

27 Dec '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1369 Summary: NPE with filesys_mapper Product: Swift Version: 0.96 Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: yadunandb(a)ci.uchicago.edu The following piece of code fails with a NullPointerException, when the file arrays out and err are using filesys_mapper. Changing the mapper to simple_mapper, gets the code running without issues. type file; int loops = toInt(arg("loops","50")); int delay = toInt(arg("delay", "1")); app (file out, file err) app_foo (file script, int delay) { bash @script delay stdout=@out stderr=@err; } file out[] <filesys_mapper; location="output", prefix="std", suffix=".out">; file err[] <filesys_mapper; location="output", prefix="std", suffix=".err">; //file out[] <simple_mapper; location="output", prefix="foo.", suffix=".out">; //file err[] <simple_mapper; location="output", prefix="foo.", suffix=".err">; file wrapper <"wrapper.sh">; foreach i in [1:loops] { tracef("Count : %i \n", i); (out[i], err[i]) = app_foo (wrapper, delay); } The logs from the run are here : http://users.rcc.uchicago.edu/~yadunand/filesys_mapper_NPE/ -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

[Bug 1267] New: Channel Timeouts with provider staging
by bugzilla-daemon＠mcs.anl.gov 01 Nov '14

01 Nov '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1267 Summary: Channel Timeouts with provider staging Product: Swift Version: 0.95 Platform: PC OS/Version: Mac OS Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: davidk(a)ci.uchicago.edu Just ran some of the provider staging tests and ran into some errors. The failing test I ran was tests/stress/IO/bagOnodes/stage_to_remoteNx2 (unmodified). The tests were running on Midway and staging data to/from MCS. It seems things are more likely to fail as the number of files and the size of the files increase. Full log at http://web.ci.uchicago.edu/~davidk/logs/channel_timeouts.log Execution failed: Exception in bash: Arguments: [wrapper.sh, dummy] Host: steamroller Directory: stage_to_remoteNx2-run001/jobs/e/bash-ekdq0xpl exception @ swift-int-staging.k, line: 181 Caused by: exception @ swift-int-staging.k, line: 177 Caused by: Error staging in file: Channel timed out. lastTime=140429-042946.162, now=140429-043146.286, channel=GSSChannel [type: client, contact: https://128.135.112.73:50004, id: 1, context: https://128.135.112.73:50004] org.globus.cog.abstraction.impl.common.execution.JobException: Error staging in file: Channel timed out. lastTime=140429-042946.162, now=140429-043146.286, channel=GSSChannel [type: client, contact: https://128.135.112.73:50004, id: 1, context: https://128.135.112.73:50004] (exit code: 520) at org.globus.cog.abstraction.coaster.service.local.JobStatusHandler.requestComplete(JobStatusHandler.java:40) at org.globus.cog.coaster.handlers.RequestHandler.receiveCompleted(RequestHandler.java:88) at org.globus.cog.coaster.channels.AbstractCoasterChannel.handleRequest(AbstractCoasterChannel.java:527) at org.globus.cog.coaster.channels.AbstractStreamCoasterChannel.stepNIO(AbstractStreamCoasterChannel.java:238) at org.globus.cog.coaster.channels.NIOMultiplexer.loop(NIOMultiplexer.java:97) at org.globus.cog.coaster.channels.NIOMultiplexer.run(NIOMultiplexer.java:56) Caused by: org.globus.cog.abstraction.impl.common.execution.JobException: Error staging in file: Channel timed out. lastTime=140429-042946.162, now=140429-043146.286, channel=GSSChannel [type: client, contact: https://128.135.112.73:50004, id: 1, context: https://128.135.112.73:50004] org.globus.cog.abstraction.impl.common.execution.JobException: Error staging in file: Channel timed out. lastTime=140429-042946.162, now=140429-043146.286, channel=GSSChannel [type: client, contact: https://128.135.112.73:50004, id: 1, context: https://128.135.112.73:50004] (exit code: 520) at org.globus.cog.abstraction.coaster.service.local.JobStatusHandler.requestComplete(JobStatusHandler.java:40) at org.globus.cog.coaster.handlers.RequestHandler.receiveCompleted(RequestHandler.java:88) at org.globus.cog.coaster.channels.AbstractCoasterChannel.handleRequest(AbstractCoasterChannel.java:527) at org.globus.cog.coaster.channels.AbstractStreamCoasterChannel.stepNIO(AbstractStreamCoasterChannel.java:238) at org.globus.cog.coaster.channels.NIOMultiplexer.loop(NIOMultiplexer.java:97) at org.globus.cog.coaster.channels.NIOMultiplexer.run(NIOMultiplexer.java:56) (exit code: 520) Progress: Tue, 29 Apr 2014 09:31:46+0000 Selecting site:11 Stage in:88 Failed:2 -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 1351] New: informative error when using Swift keyword
by bugzilla-daemon＠mcs.anl.gov 01 Nov '14

01 Nov '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1351 Summary: informative error when using Swift keyword Product: Swift Version: trunk Platform: All OS/Version: All Status: NEW Severity: normal Priority: P2 Component: Error messages AssignedTo: hategan(a)mcs.anl.gov ReportedBy: ketan(a)mcs.anl.gov CC: wozniak(a)mcs.anl.gov Just saw a user defining an app as: app (file out) myapp (file exe, file in){ bash @exe @in stdout=@out; } The error he was facing is: Could not compile SwiftScript source: line 3:24: expecting an identifier, found 'in' Replacing 'in' with '_in' fixed the error. The error message should be more explicit in saying that 'in' is a Swift keyword and can not be used as a variable (if that is the cause of the error, which I think is). Thanks, Ketan -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 1355] New: Improve error messages for errors in site.conf parsing
by bugzilla-daemon＠mcs.anl.gov 01 Nov '14

01 Nov '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=1355 Summary: Improve error messages for errors in site.conf parsing Product: Swift Version: 0.96 Platform: All OS/Version: Linux Status: ASSIGNED Severity: normal Priority: P2 Component: Configuration AssignedTo: hategan(a)mcs.anl.gov ReportedBy: wilde(a)mcs.anl.gov When an auto-generated site.conf had the following error: site.cluster { execution { type:"coaster-persistent" URL: "http://:" <---------- jobManager: "local:local" } ... I got the following cryptic error message. We should give nice precise errors with line #s about the problem, and ideally give errors for the whole file instead of quitting on the first error (if thats feasible). T$ cat run002/swift.out 2014-10-22 10:31:55,698-0400 DEBUG Loader Detailed exception: java.lang.RuntimeException: Invalid site entry 'cluster': at org.griphyn.vdl.util.SwiftConfig.site(SwiftConfig.java:614) at org.griphyn.vdl.util.SwiftConfig.build(SwiftConfig.java:401) at org.griphyn.vdl.util.SwiftConfig.load(SwiftConfig.java:255) at org.griphyn.vdl.karajan.Loader.loadConfig(Loader.java:598) at org.griphyn.vdl.karajan.Loader.main(Loader.java:148) Caused by: java.lang.NumberFormatException: For input string: "" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Integer.parseInt(Integer.java:504) at java.lang.Integer.parseInt(Integer.java:527) at org.globus.cog.abstraction.impl.common.task.ServiceContactImpl.parse(ServiceContactImpl.java:119) at org.globus.cog.abstraction.impl.common.task.ServiceContactImpl.<init>(ServiceContactImpl.java:46) at org.griphyn.vdl.util.SwiftConfig.execService(SwiftConfig.java:741) at org.griphyn.vdl.util.SwiftConfig.execution(SwiftConfig.java:707) at org.griphyn.vdl.util.SwiftConfig.site(SwiftConfig.java:587) ... 4 more Could not start execution: Invalid site entry 'cluster': : For input string: "" -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1