March 2012 - Asg - mailman3.cels.anl.gov

[aesop] #26: OSX warnings during build
by aesop 12 Sep '14

12 Sep '14

#26: OSX warnings during build -------------------------------+-------------------------------------------- Reporter: rross@… | Type: defect Status: new | Priority: don't know Component: don't know | Version: Keywords: | -------------------------------+-------------------------------------------- Lots of warnings when building on my OSX box. -- Rob --- make -C src/c-utils GHC parser/ae-blocking-parser CC opcache.o CC aesop.o DEP lookup3.d DEP triton-string.d CC resource.o DEP libev/event.d In file included from ../resource.c:1: ../src/c-utils/libev/ev.h: In function ‘ev_default_loop_uc_’: ../src/c-utils/libev/ev.h:548: warning: nested extern declaration of ‘ev_default_loop_ptr’ DEP libev/ev.d ../resource.c: In function ‘ae_get_stack’: ../resource.c:738: warning: format not a string literal and no format arguments CC hints.o CC libev/ev.o CC libev/event.o CC triton-string.o CC lookup3.o CC ae-ctl.o CC ae-init.o RANLIB libc-utils.a Linking parser/ae-blocking-parser ... ld: warning: could not create compact unwind for _ffi_call_unix64: does not use RBP or RSP based frame AECC src/aefile/aefile.ae AECC src/socket/aesocket.ae AECC resources/resourcebuilder/resourcebuilder.hae AECC resources/thread/aethread.hae AECC resources/timer/timer.hae CC resources/resourcebuilder/resourcebuilder.o CC resources/thread/aethread.o CC resources/timer/timer.o In file included from ../resources/timer/timer.c:14: ../src/c-utils/libev/ev.h: In function ‘ev_default_loop_uc_’: ../src/c-utils/libev/ev.h:548: warning: nested extern declaration of ‘ev_default_loop_ptr’ /usr/include/assert.h:79: warning: previous declaration of ‘__assert_rtn’ was here /usr/include/assert.h:79: warning: previous declaration of ‘__assert_rtn’ was here ../src/aefile/aefile.hae:23: warning: previous declaration of ‘aefile_pwrite’ was here ../src/aefile/aefile.hae:24: warning: previous declaration of ‘aefile_pread’ was here ../src/aefile/aefile.hae:22: warning: previous declaration of ‘aefile_fsync’ was here ../src/aefile/aefile.hae:23: warning: previous declaration of ‘aefile_fdatasync’ was here ../src/aefile/aefile.ae: In function ‘__ae_worker_aefile_fdatasync’: ../src/aefile/aefile.ae:88: warning: nested extern declaration of ‘fdatasync’ ../src/aefile/aefile.ae: At top level: ../src/aefile/aefile.hae:25: warning: previous declaration of ‘aefile_ftruncate’ was here ../src/aefile/aefile.hae:25: warning: previous declaration of ‘aefile_unlink’ was here /usr/include/assert.h:79: warning: previous declaration of ‘__assert_rtn’ was here /usr/include/assert.h:79: warning: previous declaration of ‘__assert_rtn’ was here /usr/include/string.h:172: warning: previous declaration of ‘swab’ was here /usr/include/stdlib.h:218: warning: previous declaration of ‘mkstemp’ was here /usr/include/stdlib.h:217: warning: previous declaration of ‘mktemp’ was here /usr/include/stdlib.h:235: warning: previous declaration of ‘setkey’ was here /usr/include/stdlib.h:341: warning: previous declaration of ‘valloc’ was here /usr/include/stdlib.h:340: warning: previous declaration of ‘suboptarg’ was here /usr/include/stdlib.h:206: warning: previous declaration of ‘getsubopt’ was here /usr/include/assert.h:79: warning: previous declaration of ‘__assert_rtn’ was here /usr/include/pthread.h:400: warning: previous declaration of ‘pthread_kill’ was here /usr/include/pthread.h:406: warning: previous declaration of ‘pthread_sigmask’ was here /usr/include/unistd.h:710: warning: previous declaration of ‘mknod’ was here ../src/c-utils/libev/ev.h: In function ‘ev_default_loop_uc_’: ../src/c-utils/libev/ev.h:548: warning: nested extern declaration of ‘ev_default_loop_ptr’ ../src/socket/aesocket.ae: At top level: ../src/socket/aesocket.hae:79: warning: previous declaration of ‘aesocket_ready’ was here ../src/socket/aesocket.hae:45: warning: previous declaration of ‘aesocket_accept’ was here ../src/socket/aesocket.hae:56: warning: previous declaration of ‘aesocket_read’ was here ../src/socket/aesocket.hae:67: warning: previous declaration of ‘aesocket_write’ was here RANLIB lib/libaesop.a mcswl174:build rross$ -- Ticket URL: <http://trac.mcs.anl.gov/projects/aesop/ticket/26> aesop <https://trac.mcs.anl.gov/projects/aesop> The Aesop Language

1 1

[aesop] #22: overriding CFLAGS in make or configure causes build to fail.
by aesop 12 Sep '14

12 Sep '14

#22: overriding CFLAGS in make or configure causes build to fail. ---------------------------+------------------------------------------------ Reporter: dkimpe | Owner: Type: defect | Status: new Priority: minor | Component: build system Version: git repository | Keywords: ---------------------------+------------------------------------------------ Configuring aesop using CFLAGS="-O2" (i.e. ./configure CFLAGS="-O2") or issuing make CFLAGS="-O2" causes the build to fail. One reason is that it overrides the -I needed to find aesop.h. This most likely is a bug in Makefile.in and/or aecc -- Ticket URL: <http://trac.mcs.anl.gov/projects/aesop/ticket/22> aesop <https://trac.mcs.anl.gov/projects/aesop> The Aesop Language

1 2

[aesop] #20: Provide some debugging helper functions for aesop
by aesop 23 Oct '13

23 Oct '13

#20: Provide some debugging helper functions for aesop ---------------------------+------------------------------------------------ Reporter: dkimpe | Owner: Type: enhancement | Status: new Priority: minor | Component: aesop functions Version: git repository | Keywords: ---------------------------+------------------------------------------------ - would be nice if we had a function to poke at aesop and dump a list of what contexts are running, what function they are in, etc. - the problem right now is if you stop an aesop program in gdb you don't see anything helpful; it is usually in the event loop - could trigger with function call, signal, rpc, or whatever -- Ticket URL: <http://trac.mcs.anl.gov/projects/aesop/ticket/20> aesop <https://trac.mcs.anl.gov/projects/aesop> The Aesop Language

1 1

[aesop] #19: Add call to feed variable=value keys to the aesop configuration system
by aesop 27 Feb '13

27 Feb '13

#19: Add call to feed variable=value keys to the aesop configuration system ---------------------------+------------------------------------------------ Reporter: dkimpe | Owner: Type: enhancement | Status: new Priority: minor | Component: aesop functions Version: git repository | Keywords: ---------------------------+------------------------------------------------ The goal is to link zeroconf to the more limited aesop equivalent. By adding a {{{processUpdate(key,value)}}} function we can call from zeroconf, we can easily link outside systems to the internal aesop configuration. The other reason for this call is to enable aesop code to change certain system parameters (such as blocking call debugging) easily. -- Ticket URL: <http://trac.mcs.anl.gov/projects/aesop/ticket/19> aesop <https://trac.mcs.anl.gov/projects/aesop> The Aesop Language

1 1

[aesop] #18: Enable non-resource code to register configuration items
by aesop 27 Feb '13

27 Feb '13

#18: Enable non-resource code to register configuration items ---------------------------+------------------------------------------------ Reporter: dkimpe | Owner: Type: enhancement | Status: new Priority: minor | Component: aesop functions Version: git repository | Keywords: ---------------------------+------------------------------------------------ With the move to resourcebuilder and branchthreader, we now have non- resource code needing to access the configuration values. Expose the configuration functionality outside of the resources. -- Ticket URL: <http://trac.mcs.anl.gov/projects/aesop/ticket/18> aesop <https://trac.mcs.anl.gov/projects/aesop> The Aesop Language

1 2

[aesop] #24: add rpc support to aesop repository
by aesop 26 Feb '13

26 Feb '13

#24: add rpc support to aesop repository ---------------------------+------------------------------------------------ Reporter: carns | Owner: Type: task | Status: new Priority: major | Component: parser Version: git repository | Keywords: ---------------------------+------------------------------------------------ This means adding the aercc translator and its network-related dependencies. Make sure that rpc/network support is optional and can be disabled at compile time. -- Ticket URL: <http://trac.mcs.anl.gov/projects/aesop/ticket/24> aesop <https://trac.mcs.anl.gov/projects/aesop> The Aesop Language

1 1

[aesop] #17: Enable run-time resource registration
by aesop 26 Feb '13

26 Feb '13

#17: Enable run-time resource registration ---------------------------+------------------------------------------------ Reporter: dkimpe | Owner: Type: enhancement | Status: new Priority: minor | Component: aesop functions Version: git repository | Keywords: ---------------------------+------------------------------------------------ Add support for 'runtime' resource registration. Some notes: - init and finalize might have to be blocking. - Have to take care not to call hard blocking functions (for example pthread_join) from within the init/finalize functions. This is needed to implement the functionality outlined below. ---------------------- - we have some problems with components now: - resources and non-resources are initialized differently in the aesop tree - not clear when to initialize what - have distinction between registering resource and initializing resource - consensus: - simplify things and make the following changes: - every component (even resources) has an explicit, directly callable init function - apps/servers/testprograms will explicitly call the init functions that they need - if a component depends on something else (like rosd -> tosd -> bdb) then the compent should initialize the things it needs - do reference counting in every init/finalize function so that we don't break when init is called twice - have resources "register" with aesop when you initialize them -- Ticket URL: <http://trac.mcs.anl.gov/projects/aesop/ticket/17> aesop <https://trac.mcs.anl.gov/projects/aesop> The Aesop Language

1 1

Are they extents, regions, dessert toppings, or thingies?
by Ward, Lee 05 Apr '12

05 Apr '12

All, Our discussion about data forks being one-byte records or a sequenced collection of bytes (or a "file", to use the generic term) confused me. It's also a topic that is fundamental and, so, I don't want to wait for the next call to get a better handle on the topic. What I understood from the call was that some want to view this "data fork" the same as a "record fork", except the records are not variable length, they are always one byte in length. Or, well, at least you get to define it that way. I wasn't enamored of this because it will be manipulated strictly as a file of bytes. No, not the "file" you are thinking of. I mean one of the alternative definitions one could find in a dictionary. Specifically: A line<http://dictionary.reference.com/browse/line> of persons or things arranged one behind another (distinguished from rank<http://dictionary.reference.com/browse/rank>). l'm thinking the "things" are bytes, of course. During the call, this generalization didn't seem to fit well with that use-case. Well, it does by definition but it forces one to map the "file" in a very nasty way, potentially, so I'm thinking "fit" as in natural to work with and practical to implement. To put it another way, it gives rise to an implementation requirement that is very, very ugly. I came to that conclusion because when I view things this way it removes the single-byte limit. It's a "file", yes, but it could be a "file" of two-byte, or any-number-bytes in the record. Which is what our record-fork is all about. Not quite, though. We have no way of manipulating multiple records. In the record-fork, such a thing didn't make sense. None of our use-cases required that. Hmm, except for punch in the record-fork but I'll come back to that. All this gave me pause. I felt that as a concept it was equivalent but practically it was sub-optimal. I was wrong. Not in my conclusion that it "gives rise to an implementation requirement that is very, very ugly" or is "sub-optimal", I was absolutely spot-on about that. What I was wrong about is that we've already crossed that "ugly" bridge. That ugly implementation has already been accepted into our model, since our atomic unit is the byte we are *already* storing extent information for the "data" fork. It's just that we assume one address maps one byte in that fork. Since we've already accepted that (we had to in order to store arbitrarily sized regions, with version information) it is no real pain to add one more field. In this case the new field would be used to indicate that all of the records in the "extent" have the same size. In the "data" fork, in the way I want to think of it, every extent record-length member is set to one. It could be two, though, or it could be millions, or more. Doesn't matter. It still works just fine. In fact ranges of records could even have different record lengths. Everything still works. I just plain don't have a leg to stand on because it's always the client that is defining the semantics, and the view too, that it wishes of the fork and we have already chosen a one-byte atomic unit and, thus, have already accepted the pain. I so lose. Sorry. My bad. This is the second time this week I've had to say that and it's getting really annoying. I'll try harder to "get it" in the future, I promise. I wish I could guarantee success. I can't, but I will try. I would still like to keep the number of calls down to a dull roar. I propose the following changes then: write address, range, recsiz, buf Where address == record-number (or key if you prefer), range == the number of records to write, recsiz == the number of bytes in each record. Note, I haven't specified version. Not because it is absent, only because it's use and meaning is unchanged. It's there, along with other things I don't think are relevant to the discussion at hand. The buffer should be at least range X recsiz bytes in length. If shorter, an error may result or one might store content one did not intend, or, well who knows what but it's not cool. We can't reliably detect this so it's best to just document the result as undefined. Punch is altered similarly. Probe will need to be modified to include the record size and where it now returns a length, it should return the size of the range. Is that just another kind of "length" really? I'm using range is some (vain?) attempt to distinguish it from a length of byes. Unnecessary? I think read is obvious from the description of write, yes? Well, almost. What happens if the requested range of records spans records of different size or includes those "virtual zeros"? I think "virtual zeros" in this case are defined as one-byte records containing zero, so the question can be simplified to just the range of records of different size if we maintain Rob's suggestion that "virtual zeros" are transferred. The server already knows the length of each record and, so, can source the store accurately, depositing bytes into the caller's buffer. The client can, certainly should, know the mapping since it allowed the store, previously. The client *should* know what it is doing because it previously allowed a write operation that stored different sized records in the fork or was careful to just never allow that in the first place. Which allows me to expand Rob's suggestion that we go ahead and "move" the virtual-zeros to we go ahead and "move" the bytes without regard to record size. Which is a little weird, though. We've asked for the record size information and we would be able to *know* it does not match at least some of the records in the region. We *could* return an error? We would be forcing the caller to use different calls for each region it is interested in, if they do not share the same record sizes. We could just move the bytes and let the client figure it out. Maybe define read to require address and byte-count instead of address, range, recsiz? In this case, what do we do if the byte-count doesn't completely map all of the bytes in the records involved? Error? Just move the first few bytes in the last record? I don't think I much like this last redefine read to take different arguments approach since it only changes how the craziness manifests and not whether it manifests. A bunch of questions that are detail. You can reply and we could talk about it on the mailing list or just wit for the next con call. I think I have preferences but not strong ones so I'd like to hear what others think. I hope by taking the time to admit my mistake, apologize, then think (a lot) and propose at least some of the modifications we might need in order to offer a robust solution I've at least partially atoned for my stupidity? In any case, I'll try to be less dense. It's hard though, it really is, I was born stupid. Thanks. --Lee

3 3

Re: [ASG] [aesop] #7: integrate library linking and paths with aecc
by aesop 30 Mar '12

30 Mar '12

#7: integrate library linking and paths with aecc ---------------------------+------------------------------------------------ Reporter: carns | Owner: dkimpe Type: defect | Status: accepted Priority: major | Component: code generator Version: git repository | Keywords: ---------------------------+------------------------------------------------ Changes (by dkimpe): * owner: => dkimpe * status: assigned => accepted Comment: I'll add a pkg-config file for that. That mechanism is already used in other context to determine which CPPFLAGS/CFLAGS and LDFLAGS are required. -- Ticket URL: <http://trac.mcs.anl.gov/projects/aesop/ticket/7#comment:6> aesop <https://trac.mcs.anl.gov/projects/aesop> The Aesop Language

1 0

Using Aesop in external projects
by Phil Carns 30 Mar '12

30 Mar '12

Just a quick Aesop update for anyone interested: For some background, the Aesop repo that we provided at the Clemson kick-off meeting did not really produce anything useful when you did "make install". You had to run aecc from the build directory and set a variety of non-intuitive CFLAGS and LDFLAGS, making it a bit of a hassle to get started with. This has been improved significantly now, and there is a new wiki page briefly describing how to properly install Aesop and use it in external projects (as opposed to just building examples in the Aesop tree): https://trac.mcs.anl.gov/projects/aesop/wiki/Using%20Aesop%20in%20other%20p… This still isn't perfect; the aecc flags are a little strange, and you still have to explicitly add the correct libraries when linking, but it at least has no dependencies on the original Aesop tree and the minimal arguments are much simpler. We can refine it further later on. -Phil

1 0