April 2012 - Asg - mailman3.cels.anl.gov

Reminder for Call / Agenda
by Dries Kimpe 05 Apr '12

05 Apr '12

Hi All, This is a reminder for the call tomorrow: Conference Access Number: 760-984-1000 Conference Passcode : 692481 # ANL : Room 7173 I've updated the document (attached): - explicitly added location / updated figure - Removed data/record fork distinction - Cleaned up terminology (attribute/record) - Fixed punch (wildcards) - Added read conditional - Removed answered/resolved discussion points Tomorrow, we'll continue the discussion: - Further define location and discuss how record/fork/object/model map to location - Discuss merge of fork types - Various clarifications/open questions Dries

2 2

aesop Repository branch, master, updated. db38d9e639f3cb955481442a7f9003b6f09c9da4
by noreply＠mcs.anl.gov 05 Apr '12

05 Apr '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "aesop Repository". The branch, master has been updated via db38d9e639f3cb955481442a7f9003b6f09c9da4 (commit) from 1dec09792c03cba518154b7cb476277b8d7e6139 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit db38d9e639f3cb955481442a7f9003b6f09c9da4 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Apr 5 11:58:35 2012 -0400 test program for #38 resource_unregister() bug ----------------------------------------------------------------------- Summary of changes: tests/module.mk.in | 3 +- tests/resource-register.c | 53 +++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+), 1 deletions(-) create mode 100644 tests/resource-register.c Diff of changes: diff --git a/tests/module.mk.in b/tests/module.mk.in index e8e3a91..4840b0f 100644 --- a/tests/module.mk.in +++ b/tests/module.mk.in @@ -1,7 +1,8 @@ DIR := tests AETESTSRC += $(DIR)/test-hints.ae - AETESTSRC += $(DIR)/blocking-overhead.ae +TESTSRC += $(DIR)/resource-register.c + diff --git a/tests/resource-register.c b/tests/resource-register.c new file mode 100644 index 0000000..e2c2f2b --- /dev/null +++ b/tests/resource-register.c @@ -0,0 +1,53 @@ +/* + * (C) 2009 The University of Chicago + * + * See COPYRIGHT in top-level directory. + */ +#include <assert.h> +#include <aesop/aesop.h> + +#include "resource.h" + +struct ae_resource rsc1 = +{ + .resource_name = "rsc1", + .poll_context = NULL, + .cancel = NULL, + .config_array = NULL, +}; + +struct ae_resource rsc2 = +{ + .resource_name = "rsc2", + .poll_context = NULL, + .cancel = NULL, + .config_array = NULL, +}; + +int main(int argc, char **argv) +{ + int ret; + int rsc1_id; + int rsc2_id; + + ret = ae_resource_register(&rsc1, &rsc1_id); + assert(ret == 0); + + ret = ae_resource_register(&rsc2, &rsc2_id); + assert(ret == 0); + + ae_resource_unregister(rsc1_id); + + ae_resource_unregister(rsc2_id); + + return 0; +} + +/* + * Local variables: + * c-indent-level: 4 + * c-basic-offset: 4 + * End: + * + * vim: ft=c ts=8 sts=4 sw=4 expandtab + */ hooks/post-receive -- aesop Repository

1 0

Are they extents, regions, dessert toppings, or thingies?
by Ward, Lee 05 Apr '12

05 Apr '12

All, Our discussion about data forks being one-byte records or a sequenced collection of bytes (or a "file", to use the generic term) confused me. It's also a topic that is fundamental and, so, I don't want to wait for the next call to get a better handle on the topic. What I understood from the call was that some want to view this "data fork" the same as a "record fork", except the records are not variable length, they are always one byte in length. Or, well, at least you get to define it that way. I wasn't enamored of this because it will be manipulated strictly as a file of bytes. No, not the "file" you are thinking of. I mean one of the alternative definitions one could find in a dictionary. Specifically: A line<http://dictionary.reference.com/browse/line> of persons or things arranged one behind another (distinguished from rank<http://dictionary.reference.com/browse/rank>). l'm thinking the "things" are bytes, of course. During the call, this generalization didn't seem to fit well with that use-case. Well, it does by definition but it forces one to map the "file" in a very nasty way, potentially, so I'm thinking "fit" as in natural to work with and practical to implement. To put it another way, it gives rise to an implementation requirement that is very, very ugly. I came to that conclusion because when I view things this way it removes the single-byte limit. It's a "file", yes, but it could be a "file" of two-byte, or any-number-bytes in the record. Which is what our record-fork is all about. Not quite, though. We have no way of manipulating multiple records. In the record-fork, such a thing didn't make sense. None of our use-cases required that. Hmm, except for punch in the record-fork but I'll come back to that. All this gave me pause. I felt that as a concept it was equivalent but practically it was sub-optimal. I was wrong. Not in my conclusion that it "gives rise to an implementation requirement that is very, very ugly" or is "sub-optimal", I was absolutely spot-on about that. What I was wrong about is that we've already crossed that "ugly" bridge. That ugly implementation has already been accepted into our model, since our atomic unit is the byte we are *already* storing extent information for the "data" fork. It's just that we assume one address maps one byte in that fork. Since we've already accepted that (we had to in order to store arbitrarily sized regions, with version information) it is no real pain to add one more field. In this case the new field would be used to indicate that all of the records in the "extent" have the same size. In the "data" fork, in the way I want to think of it, every extent record-length member is set to one. It could be two, though, or it could be millions, or more. Doesn't matter. It still works just fine. In fact ranges of records could even have different record lengths. Everything still works. I just plain don't have a leg to stand on because it's always the client that is defining the semantics, and the view too, that it wishes of the fork and we have already chosen a one-byte atomic unit and, thus, have already accepted the pain. I so lose. Sorry. My bad. This is the second time this week I've had to say that and it's getting really annoying. I'll try harder to "get it" in the future, I promise. I wish I could guarantee success. I can't, but I will try. I would still like to keep the number of calls down to a dull roar. I propose the following changes then: write address, range, recsiz, buf Where address == record-number (or key if you prefer), range == the number of records to write, recsiz == the number of bytes in each record. Note, I haven't specified version. Not because it is absent, only because it's use and meaning is unchanged. It's there, along with other things I don't think are relevant to the discussion at hand. The buffer should be at least range X recsiz bytes in length. If shorter, an error may result or one might store content one did not intend, or, well who knows what but it's not cool. We can't reliably detect this so it's best to just document the result as undefined. Punch is altered similarly. Probe will need to be modified to include the record size and where it now returns a length, it should return the size of the range. Is that just another kind of "length" really? I'm using range is some (vain?) attempt to distinguish it from a length of byes. Unnecessary? I think read is obvious from the description of write, yes? Well, almost. What happens if the requested range of records spans records of different size or includes those "virtual zeros"? I think "virtual zeros" in this case are defined as one-byte records containing zero, so the question can be simplified to just the range of records of different size if we maintain Rob's suggestion that "virtual zeros" are transferred. The server already knows the length of each record and, so, can source the store accurately, depositing bytes into the caller's buffer. The client can, certainly should, know the mapping since it allowed the store, previously. The client *should* know what it is doing because it previously allowed a write operation that stored different sized records in the fork or was careful to just never allow that in the first place. Which allows me to expand Rob's suggestion that we go ahead and "move" the virtual-zeros to we go ahead and "move" the bytes without regard to record size. Which is a little weird, though. We've asked for the record size information and we would be able to *know* it does not match at least some of the records in the region. We *could* return an error? We would be forcing the caller to use different calls for each region it is interested in, if they do not share the same record sizes. We could just move the bytes and let the client figure it out. Maybe define read to require address and byte-count instead of address, range, recsiz? In this case, what do we do if the byte-count doesn't completely map all of the bytes in the records involved? Error? Just move the first few bytes in the last record? I don't think I much like this last redefine read to take different arguments approach since it only changes how the craziness manifests and not whether it manifests. A bunch of questions that are detail. You can reply and we could talk about it on the mailing list or just wit for the next con call. I think I have preferences but not strong ones so I'd like to hear what others think. I hope by taking the time to admit my mistake, apologize, then think (a lot) and propose at least some of the modifications we might need in order to offer a robust solution I've at least partially atoned for my stupidity? In any case, I'll try to be less dense. It's hard though, it really is, I was born stupid. Thanks. --Lee

3 3

Reminder for Call / Agenda [UPDATE]
by Dries Kimpe 04 Apr '12

04 Apr '12

Now with time and date! Hi All, This is a reminder for the call tomorrow: Date : Thursday, April 5th Time : 1 PM CST (sorry, I only speak CST) Conference Access Number: 760-984-1000 Conference Passcode : 692481 # ANL : Room 7173 I've updated the document (attached): - explicitly added location / updated figure - Removed data/record fork distinction - Cleaned up terminology (attribute/record) - Fixed punch (wildcards) - Added read conditional - Removed answered/resolved discussion points Tomorrow, we'll continue the discussion: - Further define location and discuss how record/fork/object/model map to location - Discuss merge of fork types - Various clarifications/open questions Dries

2 2

SSM slides from Clemson visit
by Matthew Farmer 04 Apr '12

04 Apr '12

All, There was some interest in obtaining the slides I presented at Clemson yesterday -- attached. Enjoyed the visit; it was very productive. Thanks for having us *grin* Shane

1 0

aesop Repository branch, master, updated. 1dec09792c03cba518154b7cb476277b8d7e6139
by noreply＠mcs.anl.gov 03 Apr '12

03 Apr '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "aesop Repository". The branch, master has been updated via 1dec09792c03cba518154b7cb476277b8d7e6139 (commit) from f8755127cbf9b0d95d77aa1eae8641a9771b6b91 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 1dec09792c03cba518154b7cb476277b8d7e6139 Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Tue Apr 3 15:42:00 2012 -0500 Fix memory fault in aessm_unlink ----------------------------------------------------------------------- Summary of changes: src/ssm/aessm.ae | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) Diff of changes: diff --git a/src/ssm/aessm.ae b/src/ssm/aessm.ae index 457abd0..c897665 100644 --- a/src/ssm/aessm.ae +++ b/src/ssm/aessm.ae @@ -223,7 +223,7 @@ __blocking int aessm_unlink(ssm_me_id me_id, ssm_evt_p evt_id) rb_slot_destroy(&cb_dt->slot1); rb_slot_destroy(&cb_dt->slot2); - free(cb_id); + ssm_cb_delete(cb_id); free(cb_dt); return rc; hooks/post-receive -- aesop Repository

1 0

aesop Repository branch, master, updated. f8755127cbf9b0d95d77aa1eae8641a9771b6b91
by noreply＠mcs.anl.gov 03 Apr '12

03 Apr '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "aesop Repository". The branch, master has been updated via f8755127cbf9b0d95d77aa1eae8641a9771b6b91 (commit) via 586e1c4a592e642f36118ae59055908096b12404 (commit) from 3aa8160a8eb61d564b1cd65a3edc6cceaef3a724 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit f8755127cbf9b0d95d77aa1eae8641a9771b6b91 Merge: 586e1c4a592e642f36118ae59055908096b12404 3aa8160a8eb61d564b1cd65a3edc6cceaef3a724 Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Tue Apr 3 15:32:45 2012 -0500 Merge branch 'master' of git.mcs.anl.gov:aesop commit 586e1c4a592e642f36118ae59055908096b12404 Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Tue Apr 3 15:32:18 2012 -0500 Allow resources to have the poll function take user specified argument ----------------------------------------------------------------------- Summary of changes: parser/tests/blocking/btest-impl.c | 2 +- resource.c | 14 ++++++++++++-- resource.h | 3 ++- resources/timer/timer.c | 2 +- 4 files changed, 16 insertions(+), 5 deletions(-) Diff of changes: diff --git a/parser/tests/blocking/btest-impl.c b/parser/tests/blocking/btest-impl.c index a8e3440..9c1bb30 100644 --- a/parser/tests/blocking/btest-impl.c +++ b/parser/tests/blocking/btest-impl.c @@ -338,7 +338,7 @@ static struct btest_op * poll_list(ae_ops_t *list, int *more) return NULL; } -static int btest_poll(ae_context_t context) +static int btest_poll(ae_context_t context, void *user_data) { int more, request; struct btest_op *b; diff --git a/resource.c b/resource.c index a090dd6..314e8ee 100644 --- a/resource.c +++ b/resource.c @@ -16,8 +16,9 @@ struct ae_poll_data { ev_async async; - int (*poll_context)(ae_context_t context); + int (*poll_context)(ae_context_t context, void *user_data); ae_context_t context; + void *user_data; }; /* data structures and tables used to track information for @@ -87,7 +88,7 @@ static void ev_async_cb(EV_P_ ev_async *w, int revents) (struct ae_poll_data*)(((char*)w)-offsetof(struct ae_poll_data, async)); assert(poll_data->poll_context); - ret = poll_data->poll_context(poll_data->context); + ret = poll_data->poll_context(poll_data->context, poll_data->user_data); /* TODO: error handling? */ aesop_error_assert(ret); @@ -132,6 +133,12 @@ int ae_resource_init_register(const char* resource_name, int ae_resource_register(struct ae_resource *resource, int *newid) { + return ae_resource_register_with_data(resource, newid, NULL); +} + +int ae_resource_register_with_data(struct ae_resource *resource, int *newid, + void *user_data) +{ int reindex = ae_resource_count; int ret; @@ -156,6 +163,7 @@ int ae_resource_register(struct ae_resource *resource, int *newid) ae_resource_entries[reindex].poll_data.poll_context = resource->poll_context; ae_resource_entries[reindex].poll_data.context = NULL; + ae_resource_entries[reindex].poll_data.user_data = user_data; ev_async_start(eloop, &ae_resource_entries[reindex].poll_data.async); ae_resource_entries[reindex].id = AE_RESOURCE_IDX2ID(reindex); @@ -627,6 +635,8 @@ int _ae_context_create(ae_context_t *context, const char *format __attribute__(( c->poll_data[reindex].poll_context = ae_resource_entries[j].poll_data.poll_context; c->poll_data[reindex].context = c; + c->poll_data[reindex].user_data = + ae_resource_entries[j].poll_data.user_data; if(ae_resource_entries[j].resource->register_context) { diff --git a/resource.h b/resource.h index c539be4..5fb30ec 100644 --- a/resource.h +++ b/resource.h @@ -55,7 +55,7 @@ struct ae_resource { const char *resource_name; int (*test)(ae_op_id_t id, int ms_timeout); - int (*poll_context)(ae_context_t context); + int (*poll_context)(ae_context_t context, void *user_data); int (*cancel)(ae_context_t ctx, ae_op_id_t id); int (*register_context)(ae_context_t context); int (*unregister_context)(ae_context_t context); @@ -78,6 +78,7 @@ int ae_resource_init_all(void); /* Called by resources to register themselves to the resource framework once * they are initialized and ready for use by aesop */ int ae_resource_register(struct ae_resource *resource, int *newid); +int ae_resource_register_with_data(struct ae_resource *resource, int *newid, void *user_data); void ae_resource_unregister(int resource_id); /* Called by resources to request polling from the event loop */ diff --git a/resources/timer/timer.c b/resources/timer/timer.c index ed8fa21..adc6147 100644 --- a/resources/timer/timer.c +++ b/resources/timer/timer.c @@ -111,7 +111,7 @@ ae_define_post(int, aesop_timer, int millisecs) return AE_SUCCESS; } -static int aesop_timer_poll(ae_context_t context) +static int aesop_timer_poll(ae_context_t context, void *arg) { struct ae_op *op; struct ae_op *op_head; hooks/post-receive -- aesop Repository

1 0

Storage API meeting
by Curry, Matthew Leon 03 Apr '12

03 Apr '12

All, ANL has proposed that we have the next Storage API con call on Thursday, April 5 at 1pm CDT/12pm MDT. If you need to attend this meeting, and this time does not work for you, let's talk about alternatives. Thanks, Matthew

2 1