March 2013 - swift-bugs - mailman3.cels.anl.gov

[Bug 766] New: swift command gives Java stacktrace when run in non-writable directory
by bugzilla-daemon＠mcs.anl.gov 30 Aug '13

30 Aug '13

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=766 Summary: swift command gives Java stacktrace when run in non-writable directory Product: Swift Version: trunk Platform: PC OS/Version: Mac OS Status: ASSIGNED Severity: minor Priority: P3 Component: SwiftScript language AssignedTo: davidk(a)ci.uchicago.edu ReportedBy: wilde(a)mcs.anl.gov sandbox$ swift -version no sites file specified, setting to default: /home/wilde/swift/rev/trunk/etc/sites.xml log4j:ERROR setFile(null,true) call failed. java.io.FileNotFoundException: swift.log (Permission denied) at java.io.FileOutputStream.openAppend(Native Method) at java.io.FileOutputStream.<init>(FileOutputStream.java:188) at java.io.FileOutputStream.<init>(FileOutputStream.java:113) at org.apache.log4j.FileAppender.setFile(FileAppender.java:294) at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165) at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307) at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172) at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104) at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:809) at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:735) at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:615) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:502) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:547) at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:483) at org.apache.log4j.LogManager.<clinit>(LogManager.java:127) at java.lang.J9VMInternals.initializeImpl(Native Method) at java.lang.J9VMInternals.initialize(J9VMInternals.java:200) at org.apache.log4j.Logger.getLogger(Logger.java:117) at org.globus.cog.karajan.Loader.<clinit>(Loader.java:47) at java.lang.J9VMInternals.initializeImpl(Native Method) at java.lang.J9VMInternals.initialize(J9VMInternals.java:200) at java.lang.J9VMInternals.initialize(J9VMInternals.java:167) Swift trunk swift-r5739 cog-r3368 (cog modified locally) sandbox$ pwd /home/sinitskiy/energylandscape sandbox$ -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

[Bug 746] New: Cobalt provider on Eureka only uses 8 cores in multi-node jobs
by bugzilla-daemon＠mcs.anl.gov 30 Aug '13

30 Aug '13

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=746 Summary: Cobalt provider on Eureka only uses 8 cores in multi-node jobs Product: Swift Version: 0.93 Platform: PC OS/Version: Mac OS Status: ASSIGNED Severity: normal Priority: P2 Component: Providers AssignedTo: davidk(a)ci.uchicago.edu ReportedBy: wilde(a)mcs.anl.gov CC: hategan(a)mcs.anl.gov, davidk(a)ci.uchicago.edu, wozniak(a)mcs.anl.gov On Eureka, when I request coaster slots with nodegranularity=2 and have 8 Cobalt jobs running (for a total of 16 8-core nodes), is that I never see more than 64 jobs active. My throttle should allow 128 jobs. It looks to me like, possibly, we are only starting a coaster worker on one of the two nodes. I.e., our "multinode" logic in the submit file may be broken. Here's my sites file: <pool handle="cobalt"> <execution provider="coaster" jobmanager="local:cobalt" url="eureka.alcf.anl.gov" /> <filesystem provider="local" />  <profile namespace="globus" key="project">MTCScienceApps</profile> <profile namespace="globus" key="queue">default</profile> <profile namespace="karajan" key="jobthrottle">1.28</profile> <profile namespace="karajan" key="initialScore">10000</profile> <profile namespace="globus" key="jobsPerNode">8</profile>   <profile namespace="globus" key="slots">8</profile> <profile namespace="globus" key="maxTime">7200</profile>  <profile namespace="globus" key="maxWallTime">00:10:00</profile> <profile namespace="globus" key="nodeGranularity">2</profile> <profile namespace="globus" key="maxNodes">2</profile> <workdirectory>/home/wilde/LZ/run012/swiftwork</workdirectory> </pool> -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

[Bug 961] New: Wrapper staging file not found
by bugzilla-daemon＠mcs.anl.gov 11 Jun '13

11 Jun '13

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=961 Summary: Wrapper staging file not found Product: Swift Version: trunk Platform: PC OS/Version: Mac OS Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: hategan(a)mcs.anl.gov ReportedBy: davidk(a)ci.uchicago.edu I just tried a quick test of wrapper staging in trunk, but am getting an error that the input file is not found. The error is: Caused by: Cannot stage in localhost/data.txt. File not found. The full log is at http://www.ci.uchicago.edu/~davidk/logs/catsn-20130305-1113-r1xsgfcc.log. I'm not sure if I have things set up correctly or not. Here is the setup I am testing with. cf -- wrapperlog.always.transfer=true sitedir.keep=true execution.retries=0 lazy.errors=true status.mode=provider use.provider.staging=false provider.staging.pin.swiftfiles=false use.wrapper.staging=true wrapper.staging.local.server=file:// sites.xml -------- <config> <pool handle="localhost"> <execution provider="local" url="none"/> <profile namespace="karajan" key="jobThrottle">0</profile> <profile namespace="karajan" key="initialScore">10000</profile> <filesystem provider="local"/> <workdirectory>/home/davidk/swiftwork</workdirectory> <scratch>/scratch/local/davidk</scratch> </pool> </config> catsn.swift --------- type file; app (file o) cat (file i) { cat @i stdout=@o; } file out[]<simple_mapper; location=".", prefix="catsn.",suffix=".out">; foreach j in [1:@toInt(@arg("n","10"))] { file data <"data.txt">; out[j] = cat(data); } -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 4

[Bug 947] New: Swift gives hang-checker message on missing data condition
by bugzilla-daemon＠mcs.anl.gov 11 Apr '13

11 Apr '13

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=947 Summary: Swift gives hang-checker message on missing data condition Product: Swift Version: 0.94 Platform: PC OS/Version: Mac OS Status: ASSIGNED Severity: normal Priority: P2 Component: Error messages AssignedTo: hategan(a)mcs.anl.gov ReportedBy: wilde(a)mcs.anl.gov CC: wozniak(a)mcs.anl.gov >From a DSSAT incident, per David: " The grid list contains a list of relative grids, like "224/224001" It reads that data and then tried to open a file with something like weatherprefix/$griddata and scenarioprefix/$griddata If those directories don't exist it looks like a hang because swift waits for them to be created To get around that I created a utility that fixes it, but I don't think it was used here." Can we instead give a more clear, direct message of what happened here? Or, in the user guide, show the user how to work backwards from the message to the diagnosis and fix? -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

[Bug 969] New: Default values for sites catalogue
by bugzilla-daemon＠mcs.anl.gov 19 Mar '13

19 Mar '13

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=969 Summary: Default values for sites catalogue Product: Swift Version: trunk Platform: All OS/Version: Linux Status: NEW Severity: normal Priority: P2 Component: General AssignedTo: ketan(a)mcs.anl.gov ReportedBy: ketan(a)mcs.anl.gov CC: davidk(a)ci.uchicago.edu Sites catalog values preset to defaults. Ideally most, but in particular, the workdirectory element be set to a directory named after the script in format similar to .d directories: scriptname-xxxx.wd in cwd. -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

[Bug 481] Hang when accessing nonexistent array element
by bugzilla-daemon＠mcs.anl.gov 18 Mar '13

18 Mar '13

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=481 Mihael Hategan <hategan(a)mcs.anl.gov> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED --- Comment #1 from Mihael Hategan <hategan(a)mcs.anl.gov> 2013-03-18 19:47:21 --- Not a trivial issue, but a tentative fix exists in the faster branch. I would rather not port it to 0.94 because I'm afraid it would unnecessarily delay the release. The issue is that when an array item is accessed (whether for reading or writing), an open node is created. If the only code path that accesses that array item is the one that tries to read from it, that node will never be closed properly. There are two naive immediate solutions: 1. deep close the array based on write reference counting, but that will cause t[4] to be closed with no value, breaking the contract that a closed node is valid. 2. Avoid the creation of a node from a read operation, and add a listener to the array instead. This has the difficulty that, in a valid program, even though such a node may be written early in the game, the value may never become useable until the entire array is closed. The alternative of filtering listeners based on the exact index has various technical difficulties. The solution committed in r6399 employs a modified version of (1): arrays are now deep-closed based on write reference counting, but all leaf data must additionally have an actual value stored. In the case of primitive types, this is the actual data, and in the case of mapped types, a specific token is used. When waiting for a variable or when accessing its value, a run-time exception is thrown if the value is not set. I believe that this is still not complete since compound types are still allowed to not have a value, but the same problem can potentially occur in the above example if the array is an array of compound types. So the bug stays open for now. -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 0

[Bug 602] java.xml file in working dir causes swift to fail
by bugzilla-daemon＠mcs.anl.gov 18 Mar '13

18 Mar '13

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=602 Mihael Hategan <hategan(a)mcs.anl.gov> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |RESOLVED Resolution| |FIXED --- Comment #3 from Mihael Hategan <hategan(a)mcs.anl.gov> 2013-03-18 04:00:16 --- Fixed in 0.94 cog r3644. -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 0

[Bug 397] Swift hangs when GridFTP servers fail to respond to connect requests
by bugzilla-daemon＠mcs.anl.gov 18 Mar '13

18 Mar '13

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=397 Mihael Hategan <hategan(a)mcs.anl.gov> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|ASSIGNED |RESOLVED Resolution| |FIXED --- Comment #3 from Mihael Hategan <hategan(a)mcs.anl.gov> 2013-03-18 03:37:26 --- Did what could be done on the swift side. If gridftp servers have problems with multiple concurrent connections, throttling should be used to address the issue. This should, perhaps be documented. -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 0

[Bug 253] Enhance coaster timeout processing in passive and persistent modes
by bugzilla-daemon＠mcs.anl.gov 18 Mar '13

18 Mar '13

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=253 Mihael Hategan <hategan(a)mcs.anl.gov> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |RESOLVED Resolution| |FIXED -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 0

[Bug 209] 1MB file corrupted when transferred using Coaster-Data-Provider
by bugzilla-daemon＠mcs.anl.gov 18 Mar '13

18 Mar '13

https://bugzilla.mcs.anl.gov/swift/show_bug.cgi?id=209 Mihael Hategan <hategan(a)mcs.anl.gov> changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |skenny(a)uchicago.edu --- Comment #2 from Mihael Hategan <hategan(a)mcs.anl.gov> 2013-03-18 03:32:32 --- *** Bug 222 has been marked as a duplicate of this bug. *** -- Configure bugmail: https://bugzilla.mcs.anl.gov/swift/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 0