February 2013 - swift-bugs - mailman3.cels.anl.gov

[Bug 945] New: coaster.service.job.manager.SortedJobSet NoSuchElementException
by bugzilla-daemon＠mcs.anl.gov 30 Jul '15

30 Jul '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=945 Summary: coaster.service.job.manager.SortedJobSet NoSuchElementException Product: Swift Version: trunk Platform: PC OS/Version: Mac OS Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: davidk(a)ci.uchicago.edu I am seeing this error on Beagle: CoasterService fatal error: Exception caught in block processor java.util.NoSuchElementException at org.globus.cog.abstraction.coaster.service.job.manager.SortedJobSet$1.next(SortedJobSet.java:182) at org.globus.cog.abstraction.coaster.service.job.manager.SortedJobSet$1.next(SortedJobSet.java:162) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.allocateBlocks(BlockQueueProcessor.java:609) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.updatePlan(BlockQueueProcessor.java:734) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.run(BlockQueueProcessor.java:139) Exception caught in block processor java.util.NoSuchElementException at org.globus.cog.abstraction.coaster.service.job.manager.SortedJobSet$1.next(SortedJobSet.java:182) at org.globus.cog.abstraction.coaster.service.job.manager.SortedJobSet$1.next(SortedJobSet.java:162) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.allocateBlocks(BlockQueueProcessor.java:609) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.updatePlan(BlockQueueProcessor.java:734) at org.globus.cog.abstraction.coaster.service.job.manager.BlockQueueProcessor.run(BlockQueueProcessor.java:139) The full log is at http://www.ci.uchicago.edu/~davidk/logs/modis-20130130-1819-jrybzslc.log -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 9

[Bug 714] New: Give more information for "could not convert" error message
by bugzilla-daemon＠mcs.anl.gov 19 Jun '15

19 Jun '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=714 Summary: Give more information for "could not convert" error message Product: Swift Version: 0.93 Platform: PC OS/Version: Mac OS Status: NEW Severity: normal Priority: P2 Component: SwiftScript language AssignedTo: davidk(a)ci.uchicago.edu ReportedBy: wilde(a)mcs.anl.gov Swift dies with the following message: Execution failed: Could not convert value to number: nan Provide at least the built-in function name and ideally the line number of the failing operation. -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

[Bug 710] New: Keyword argument passing doesnt seem to work as described in user guide
by bugzilla-daemon＠mcs.anl.gov 06 Mar '15

06 Mar '15

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=710 Summary: Keyword argument passing doesnt seem to work as described in user guide Product: Swift Version: 0.93 Platform: PC OS/Version: Mac OS Status: NEW Severity: minor Priority: P4 Component: SwiftScript language AssignedTo: hategan(a)mcs.anl.gov ReportedBy: wilde(a)mcs.anl.gov (need to provide test cases) -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

[Bug 913] New: Large coaster block on beagle is missing workers
by bugzilla-daemon＠mcs.anl.gov 19 Oct '14

19 Oct '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=913 Summary: Large coaster block on beagle is missing workers Product: Swift Version: 0.94 Platform: PC OS/Version: Mac OS Status: ASSIGNED Severity: blocker Priority: P2 Component: Providers AssignedTo: hategan(a)mcs.anl.gov ReportedBy: wilde(a)mcs.anl.gov CC: hategan(a)mcs.anl.gov, davidk(a)ci.uchicago.edu, wozniak(a)mcs.anl.gov A Swift run using a single 520-node block of workers on Beagle only sees 503 workers. This is running 1 worker per node, 1 job per node (running 24-core OpenMP jobs). Only 503 worker connections seem to be seen by Swift. Logs are on Beagle in: /lustre/beagle/wilde/scicol/svn/run028 An earlier 420-node run with a similar configuration saw all its workers and ran the expected 420 concurrent jobs. -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 893] New: Consider an ext mapper for arrays that takes a full command string
by bugzilla-daemon＠mcs.anl.gov 19 Oct '14

19 Oct '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=893 Summary: Consider an ext mapper for arrays that takes a full command string Product: Swift Version: 0.94 Platform: PC OS/Version: Mac OS Status: NEW Severity: enhancement Priority: P2 Component: SwiftScript language AssignedTo: wozniak(a)mcs.anl.gov ReportedBy: wilde(a)mcs.anl.gov (we may want to defer this to Swift/T) -- Per a swift-support thread with Neil Best: I need to qualify what I said in the prior posting. Per the User Guide, an ext mapper needs to return two columns for the mapping, so for this format you almost always need a wrapper script to produce that format. Neil, your suggestion would work for a mapper that mapped N elements of an array, with column 1 implicitly being [0], [1], ... With that caveat, your suggestion makes sense for some other new but useful mapper type. In Swift/T ( ) the approach to mappers is evolving somewhat differently, and we will discuss that with the community and provide some kind of backward compatibility option or conversion guide. You might want to do this as follows, instead: - declare "sh" as an app. - pass your find command to the sh as the value of the -c argument, and have the app return stdout as its result. - pass that to readData() into an array, and map the dataset with this array using the array_mapper: string s[] = [ "a.txt", "b.txt", "c.txt" ]; file f[] <array_mapper;files=s>; A simple example of the above belongs in the User Guide. I'll create a ticket for that as well. - Mike ----- Original Message ----- > From: "Michael Wilde" <wilde(a)mcs.anl.gov> > To: "Neil Best" <nbest(a)ci.uchicago.edu> > Cc: swift-user(a)ci.uchicago.edu > Sent: Wednesday, December 5, 2012 6:46:51 PM > Subject: Re: [Swift-user] Unix find as external mapper > Neil, > > While your suggested syntax sounds reasonable, thats not how the ext > mapper currently works. You do indeed need to create a small wrapper > script for such situations. The exec parameter is only used to specify > the pathname of the mapper script (absolute or relative to the working > dir in which you are running the swift command). > > I'll file an enhancement ticket to record your suggestion, which in > hindsight seems obvious, and which I like :) > > Thanks, > > - Mike > > ----- Original Message ----- > > From: "Neil Best" <nbest(a)ci.uchicago.edu> > > To: swift-user(a)ci.uchicago.edu > > Sent: Wednesday, December 5, 2012 4:17:22 PM > > Subject: [Swift-user] Unix find as external mapper > > I tried to use find as an external mapper in a way that seemed > > fairly > > natural to me: > > > > file monthly[] <ext; exec="find data/nc -mindepth 1 -type d -printf > > '[%f] %p.nc\n'\|.txt'">; > > > > When I run this find by itself at the command line the output looks > > like this: > > > > $ find data/nc -mindepth 1 -type d -printf '[%f] %p.nc\n' | head > > [197901] data/nc/197901.nc > > [197903] data/nc/197903.nc > > [197904] data/nc/197904.nc > > [197902] data/nc/197902.nc > > [197905] data/nc/197905.nc > > [197906] data/nc/197906.nc > > [197907] data/nc/197907.nc > > [197908] data/nc/197908.nc > > [197909] data/nc/197909.nc > > [197910] data/nc/197910.nc > > > > The find actually looks at directories. These .nc files don't exist > > yet, but each one will be an aggregation of the .nc files within the > > folder of the same name using this command: > > > > cdo mergetime data/nc/197901/*.single.nc data/nc/197901.nc > > > > I thought I could then do a nested foreach over years and months to > > execute the invocations. > > > > I presume that the external mapper is treating the exec argument as > > a > > command name and not parsing the arguments. Using the other > > parameters in the ext construct would force me to write a wrapper > > script for the find invocation since all of the arguments must have > > names apparently. > > > > Do I have this right? Maybe there is a more straightforward > > approach. > > _______________________________________________ > > Swift-user mailing list > > Swift-user(a)ci.uchicago.edu > > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user > > -- > Michael Wilde > Computation Institute, University of Chicago > Mathematics and Computer Science Division > Argonne National Laboratory > > _______________________________________________ > Swift-user mailing list > Swift-user(a)ci.uchicago.edu > https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user -- Michael Wilde Computation Institute, University of Chicago Mathematics and Computer Science Division Argonne National Laboratory _______________________________________________ Swift-user mailing list Swift-user(a)ci.uchicago.edu https://lists.ci.uchicago.edu/cgi-bin/mailman/listinfo/swift-user -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

[Bug 883] New: DSSAT coaster problem: Trying to bind invalid channel to MetaChannel
by bugzilla-daemon＠mcs.anl.gov 18 Oct '14

18 Oct '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=883 Summary: DSSAT coaster problem: Trying to bind invalid channel to MetaChannel Product: Swift Version: trunk Platform: PC OS/Version: Mac OS Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: davidk(a)ci.uchicago.edu I was trying to do a full scale run of DSSAT last night using OSG resources. Approximately 12K tasks completed successfully before I saw this error: Caused by: Could not submit job Caused by: Failed to configure channel Caused by: Could not register channel Caused by: Trying to bind invalid channel (TCP-null) to MetaChannel[cps-48722] -> BufferingChannel RunDSSAT, RunDSSAT.swift, line 26 The swift and coaster logs are pretty large - they can be found at /scratch/local/davidk/DSSAT/run007 on Communicado. -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 9

[Bug 877] New: Errors in new "Execution Failed" error messages
by bugzilla-daemon＠mcs.anl.gov 18 Oct '14

18 Oct '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=877 Summary: Errors in new "Execution Failed" error messages Product: Swift Version: trunk Platform: PC OS/Version: Mac OS Status: ASSIGNED Severity: normal Priority: P2 Component: Error messages AssignedTo: hategan(a)mcs.anl.gov ReportedBy: wilde(a)mcs.anl.gov CC: wozniak(a)mcs.anl.gov After "fixing" 875, Im now seeing different behavior from runtime error messages. I was testing an app thats a .sh script which returns exit code 1 in some situations. With provider staging, I get an error that indicates an exit 1 was returned, but then the error formatter (or something else after the error) produces a secondary error: one or more messages of the form "Attempted to unregister unregistered handler with id 8293": mid$ swift -config cfps -tc.file tc2 -sites.file midway2.xml checksite.swift -n=100 Swift trunk swift-r6074 cog-r3517 RunID: 20121122-0430-y1fexsmg Progress: time: Thu, 22 Nov 2012 04:30:43 +0000 Progress: time: Thu, 22 Nov 2012 04:30:44 +0000 Submitted:172 Active:16 Stage out:10 Finished successfully:2 Execution failed: Exception in checksand: Arguments: [] Host: sandyb Directory: checksite-20121122-0430-y1fexsmg/jobs/l/checksand-lyukbb1l Caused by: Application /home/wilde/testmpi/checksand.sh failed with an exit code of 1 checksand, checksite.swift, line 15 foreach, checksite.swift, line 14 Attempted to unregister unregistered handler with id 8293 Attempted to unregister unregistered handler with id 8291 Attempted to unregister unregistered handler with id 8295 Attempted to unregister unregistered handler with id 8297 Attempted to unregister unregistered handler with id 8299 mid$ When I set provider staging false, I dont get the secondary message, but neither do I get the "failed with exit code 1" message: mid$ swift -config cfps -tc.file tc2 -sites.file midway2.xml checksite.swift -n=100 Swift trunk swift-r6074 cog-r3517 RunID: 20121122-0435-0b97nf82 Progress: time: Thu, 22 Nov 2012 04:35:59 +0000 Progress: time: Thu, 22 Nov 2012 04:36:00 +0000 Submitted:188 Active:11 Checking status:1 Progress: time: Thu, 22 Nov 2012 04:36:01 +0000 Submitted:104 Active:56 Checking status:14 Stage out:2 Finished successfully:24 Progress: time: Thu, 22 Nov 2012 04:36:02 +0000 Submitted:29 Active:55 Checking status:42 Stage out:42 Finished successfully:25 Failed but can retry:7 Progress: time: Thu, 22 Nov 2012 04:36:03 +0000 Checking status:48 Stage out:103 Finished successfully:42 Failed but can retry:7 Execution failed: Exception in checksand: Arguments: [] Host: sandyb Directory: checksite-20121122-0435-0b97nf82/jobs/9/checksand-9j8sbb1l stderr.txt: stdout.txt: np=12 Caused by: File not found: /scratch/midway/wilde/testmpi/checksite-20121122-0435-0b97nf82/shared/outdir/s.0049.out checksand, checksite.swift, line 15 foreach, checksite.swift, line 14 mid$ -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2

[Bug 867] New: Tracing
by bugzilla-daemon＠mcs.anl.gov 18 Oct '14

18 Oct '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=867 Summary: Tracing Product: Swift Version: trunk Platform: PC OS/Version: Linux Status: NEW Severity: enhancement Priority: Medium Component: Design Decision AssignedTo: hategan(a)mcs.anl.gov ReportedBy: hategan(a)mcs.anl.gov This is to keep track and discuss tracing until it's finalized. A few general things about it: - it's not supposed to be fast; instead the aim is to provide information about what happens at the source code level to help with troubleshooting scripts - it should not affect performance when it's disabled; this is currently done in a log4j style, by wrapping tracing calls in a fast isEnabled test. The format: In the log, trace lines will appear with "TRACE" as source. This way a trace can be quickly extracted with grep. The general format is (literals are unquoted): {sourceType}, line {line}, thread {thread}, {info} where: {sourceType} - one of: DECLARE, VARIABLE, ASSIGN, ITERATE, APPCALL, CALL, SWIFTCALL, FOREACH, ITERATION or a swiftscript function (such as 'swiftscript:readData'). {line} - the line in the source code associated with this trace line. {thread} - the karajan level thread. The thread name is made of tokens separated by '-'. A thread A is a parent of thread B if the name of B is the name of A with a token added to the end. For example A = "0-0-22" is the parent of "0-0-22-1". {info} - varies by type as explained below: DECLARE - {variableName} [<{mapperType}; [{paramName1}={paramValue1}[, {paramName2}={paramValue2}[, ...]]]> ASSIGN - {variableName} = {unwrappedHandle} {unwrappedHandle} ca be: 1. a primitive value (if known at the time the line is printed) 2. a file name in the form <"{name}"> if the source is a non-composite mapped type 3. the variable name if the source is a closed composite type 4. the variable name prefixed by "?" if the source is open ITERATE - {iterationVariableName} = {value} FOREACH - no info ITERATION - [{keyVariableName} = {key}, ]{valueVariableName} = {value} Whether the key part appears depends on whether the 'foreach v, k' or 'foreach v' variant was used APPCALL 1. {procName}([{argName1} = {unwrappedHandle}[, {argName2} = ...]]) - logged when an app is invoked. {unwrappedHandle} is as explained above. 2. {procName} WAIT {variable} - logged when an app needs to wait for a variable CALL {procName}([{argName1} = {unwrappedHandle}[, {argName2} = ...]]) - for compound procedure calls SWIFTCALL {procName}([{argName1} = {unwrappedHandle}[, {argName2} = ...]]) - for swift internal function calls (such as readData). Few of these are logged currently VARIABLE 1. {varName} INITIALIZED - when a variable is initialized (i.e. all the mapping dependencies have been satisfied) 2. {varName} WAIT {var2} - when a variable needs to wait for a variable that is part of a mapping dependency -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 3

[Bug 801] New: Built-in mappers should give an error when an unrecognized parameter is specified.
by bugzilla-daemon＠mcs.anl.gov 18 Oct '14

18 Oct '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=801 Summary: Built-in mappers should give an error when an unrecognized parameter is specified. Product: Swift Version: 0.93 Platform: All OS/Version: All Status: NEW Severity: normal Priority: P2 Component: SwiftScript language AssignedTo: davidk(a)ci.uchicago.edu ReportedBy: wilde(a)mcs.anl.gov >From user R. Weiss: "So like I said, it turns out this whole issue boils down to a typo. I would suggest that swift should throw up some sort of warning if you pass something to one of the pre-defined mappers that is not defined as one of its parameters. I would have expected swift to complain that "sufix" is an unknown parameter of filesys_mapper." -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

[Bug 752] New: Coaster provider does not retry worker launches on qsub failures
by bugzilla-daemon＠mcs.anl.gov 18 Oct '14

18 Oct '14

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=752 Summary: Coaster provider does not retry worker launches on qsub failures Product: Swift Version: 0.93 Platform: PC OS/Version: Mac OS Status: ASSIGNED Severity: normal Priority: P2 Component: Providers AssignedTo: hategan(a)mcs.anl.gov ReportedBy: wilde(a)mcs.anl.gov CC: hategan(a)mcs.anl.gov, davidk(a)ci.uchicago.edu, wozniak(a)mcs.anl.gov If a qsub to start a worker.pl job fails (in this case, due to the user having exceeded the maximum number of concurrently allowed job submissions, 32 on Eureka) then the coaster provider never tries again to start workers for those coaster slots. Hence, when the user is running multiple swift runs concurrently, and they fluctuate in the number of active workers, the swift run that encounters the worker failures stays forever at a very low active worker count. In the current situation, it stays so low (eg 1 worker out of an expected 32) that the run needs to be killed and restarted in order to get a sufficient number of workers. Logs are on eureka in /home/wilde/LZ/runNNN where NNN is between 020 and 058. -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1