July 2013 - Triton-commits - mailman3.cels.anl.gov

branch, devel-mercury, updated. 9971e4b492f4fb39bc0ee6deb7048a7a4d6f6eb5
by noreply＠mcs.anl.gov 31 Jul '13

31 Jul '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, devel-mercury has been updated via 9971e4b492f4fb39bc0ee6deb7048a7a4d6f6eb5 (commit) from 229351495c9429cd8bd3db518478da4e485ec17e (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 9971e4b492f4fb39bc0ee6deb7048a7a4d6f6eb5 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jul 31 17:31:46 2013 -0400 cut network layer, disable compile of affected code ----------------------------------------------------------------------- Summary of changes: code/configure.ac | 6 - code/src/common/module.mk.in | 11 +- code/src/net/fault-injector/fault-method.ae | 570 ------ code/src/net/fault-injector/fault-method.h | 9 - code/src/net/fault-injector/module.mk.in | 7 - code/src/net/module.mk.in | 7 - code/src/net/mpi/module.mk.in | 10 - code/src/net/mpi/mpi-method.ae | 1943 -------------------- code/src/net/mpi/mpi-method.h | 11 - .../net/mpi/one_sided_interface_mpiimpl_2/README | 6 - .../net/mpi/one_sided_interface_mpiimpl_2/build | 1 - .../net/mpi/one_sided_interface_mpiimpl_2/mock.c | 16 - .../net/mpi/one_sided_interface_mpiimpl_2/mock.h | 134 -- .../mpi-message-method.c | 1433 --------------- .../net/mpi/one_sided_interface_mpiimpl_2/test.c | 810 -------- .../triton-message-method.h | 267 --- code/src/net/mpi/tests/module.mk.in | 12 - code/src/net/mpi/tests/noop-bench.ae | 246 --- code/src/net/ssm/module.mk.in | 9 - code/src/net/ssm/ssm-method.ae | 1723 ----------------- code/src/net/ssm/ssm-method.h | 8 - code/src/net/ssm/ssm-tcp-transport.c | 161 -- code/src/net/ssm/ssm-transport.h | 69 - code/src/net/tests/README_FOR_ONE_SIDED | 65 - code/src/net/tests/module.mk.in | 18 - code/src/net/tests/mpi-client-server.ae | 100 - code/src/net/tests/mpi-one-sided.ae | 1024 ----------- code/src/net/tests/mpi-server.ae | 45 - code/src/net/tests/sample_one_sided_test_input | 53 - code/src/net/tests/ssm-one-sided.ae | 1161 ------------ code/src/net/tests/ssm-sr-test.ae | 247 --- code/src/net/triton-addr.h | 30 - code/src/net/triton-message-method.hae | 244 --- code/src/net/triton-message.ae | 666 ------- code/src/net/triton-message.hae | 269 --- code/src/remote/module.mk.in | 13 +- 36 files changed, 14 insertions(+), 11390 deletions(-) delete mode 100644 code/src/net/fault-injector/fault-method.ae delete mode 100644 code/src/net/fault-injector/fault-method.h delete mode 100644 code/src/net/fault-injector/module.mk.in delete mode 100644 code/src/net/module.mk.in delete mode 100644 code/src/net/mpi/module.mk.in delete mode 100644 code/src/net/mpi/mpi-method.ae delete mode 100644 code/src/net/mpi/mpi-method.h delete mode 100644 code/src/net/mpi/one_sided_interface_mpiimpl_2/README delete mode 100755 code/src/net/mpi/one_sided_interface_mpiimpl_2/build delete mode 100644 code/src/net/mpi/one_sided_interface_mpiimpl_2/mock.c delete mode 100644 code/src/net/mpi/one_sided_interface_mpiimpl_2/mock.h delete mode 100644 code/src/net/mpi/one_sided_interface_mpiimpl_2/mpi-message-method.c delete mode 100644 code/src/net/mpi/one_sided_interface_mpiimpl_2/test.c delete mode 100644 code/src/net/mpi/one_sided_interface_mpiimpl_2/triton-message-method.h delete mode 100644 code/src/net/mpi/tests/module.mk.in delete mode 100644 code/src/net/mpi/tests/noop-bench.ae delete mode 100644 code/src/net/ssm/module.mk.in delete mode 100644 code/src/net/ssm/ssm-method.ae delete mode 100644 code/src/net/ssm/ssm-method.h delete mode 100644 code/src/net/ssm/ssm-tcp-transport.c delete mode 100644 code/src/net/ssm/ssm-transport.h delete mode 100644 code/src/net/tests/README_FOR_ONE_SIDED delete mode 100644 code/src/net/tests/module.mk.in delete mode 100644 code/src/net/tests/mpi-client-server.ae delete mode 100644 code/src/net/tests/mpi-one-sided.ae delete mode 100644 code/src/net/tests/mpi-server.ae delete mode 100644 code/src/net/tests/sample_one_sided_test_input delete mode 100644 code/src/net/tests/ssm-one-sided.ae delete mode 100644 code/src/net/tests/ssm-sr-test.ae delete mode 100644 code/src/net/triton-addr.h delete mode 100644 code/src/net/triton-message-method.hae delete mode 100644 code/src/net/triton-message.ae delete mode 100644 code/src/net/triton-message.hae Diff of changes: diff --git a/code/configure.ac b/code/configure.ac index 5f284df..79b4e3b 100644 --- a/code/configure.ac +++ b/code/configure.ac @@ -286,11 +286,6 @@ src/common/module.mk src/common/tests/module.mk src/common/resources/scheduling/module.mk src/common/resources/scheduling/tests/module.mk -src/net/module.mk -src/net/tests/module.mk -src/net/mpi/module.mk -src/net/mpi/tests/module.mk -src/net/fault-injector/module.mk src/remote/module.mk src/remote/tests/module.mk src/transactional-osd/module.mk @@ -320,7 +315,6 @@ doc/resilience/resilience-book.txt doc/resilience/prototype-2012-07/module.mk doc/resilience/simulation-2012-10/module.mk src/asg/module.mk -src/net/ssm/module.mk src/examples/parallel-histogram/module.mk src/examples/block-read-modify-write/module.mk ]) diff --git a/code/src/common/module.mk.in b/code/src/common/module.mk.in index 70e1f15..2e9e544 100644 --- a/code/src/common/module.mk.in +++ b/code/src/common/module.mk.in @@ -6,11 +6,14 @@ LIBSRC += $(DIR)/triton-error.c \ $(DIR)/triton-notify.c \ $(DIR)/node.c -AELIBSRC += $(DIR)/traffic-cop.ae \ - $(DIR)/triton-bootstrap.ae +#AELIBSRC += \ +# $(DIR)/triton-bootstrap.ae +# $(DIR)/traffic-cop.ae \ -AESOP_HDR += $(DIR)/traffic-cop.hae \ - $(DIR)/triton-bootstrap.hae +#AESOP_HDR += \ +# $(DIR)/triton-bootstrap.hae + +# $(DIR)/traffic-cop.hae \ COMMONDIR := src/common $(COMMONDIR)/triton-error-defs.c $(COMMONDIR)/triton-error-decls.h: $(COMMONDIR)/triton-error.spec diff --git a/code/src/net/fault-injector/fault-method.ae b/code/src/net/fault-injector/fault-method.ae deleted file mode 100644 index 2089663..0000000 --- a/code/src/net/fault-injector/fault-method.ae +++ /dev/null @@ -1,570 +0,0 @@ -#include <stdlib.h> -#include <sys/time.h> - -#include "src/net/fault-injector/fault-method.h" -#include <triton-hash.h> -#include "src/net/triton-addr.h" -#include "src/net/triton-message.hae" -#include "src/net/triton-message-method.hae" -#include <aesop/timer.hae> -#include "src/common/triton-debug.h" -#include "src/zeroconf/zeroconf.h" - -static triton_mutex_t module_lock = TRITON_MUTEX_INITIALIZER; -static int module_refcount = 0; - -#define FAULT_CONFKEY_PREFIX "triton.net.fault_injector" - -static const char *base_method_confkey = FAULT_CONFKEY_PREFIX ".base_method"; -static const char *insert_send_timeouts_at_rate_confkey = FAULT_CONFKEY_PREFIX ".insert_send_timeouts_at_rate"; -static const char *insert_recv_timeouts_at_rate_confkey = FAULT_CONFKEY_PREFIX ".insert_recv_timeouts_at_rate"; -static const char *insert_send_errors_at_rate_confkey = FAULT_CONFKEY_PREFIX ".insert_send_errors_at_rate"; -static const char *insert_recv_errors_at_rate_confkey = FAULT_CONFKEY_PREFIX ".insert_recv_errors_at_rate"; - -static struct triton_msg_method* base_method = NULL; -static float insert_send_timeouts_at_rate = 0; -static float insert_recv_timeouts_at_rate = 0; -static float insert_send_errors_at_rate = 0; -static float insert_recv_errors_at_rate = 0; -#define LONG_TIME 172800000 /* 48 hours */ -static triton_debug_mask_t fault_dbg_mask; -static triton_mutex_t fault_mutex = TRITON_MUTEX_INITIALIZER; - -/* check to see if a given message should time out */ -static int should_fail(float rate) -{ - float threshold; - - if(rate == 0) - return 0; - if(rate >= 100) - return 1; - - threshold = random(); - threshold /= (float)RAND_MAX; - threshold *= 100.0; - - triton_debug(fault_dbg_mask, "Random threshold: %f on rate %f \n", (double)threshold, (double)rate); - if(threshold < rate) - return 1; - else - return 0; -} - -static __blocking triton_msg_method_addr_t fault_addr_self(void) -{ - return base_method->addr_self(); -} - -static int fault_addr_equal(triton_msg_method_addr_t addr1, triton_msg_method_addr_t addr2) -{ - return(base_method->addr_equal(addr1, addr2)); -} - -static triton_ret_t fault_addr_to_string(triton_msg_method_addr_t node, triton_string_t *nodestr) -{ - return(base_method->addr_to_string(node, nodestr)); -} - -static triton_ret_t fault_addr_lookup(const char *name, triton_msg_method_addr_t *addr) -{ - return(base_method->addr_lookup(name, addr)); -} - -static triton_ret_t fault_addr_free(triton_msg_method_addr_t addr) -{ - return(base_method->addr_free(addr)); -} - -static triton_ret_t fault_addr_copy(const triton_msg_method_addr_t orig_addr, triton_msg_method_addr_t *copy) -{ - return(base_method->addr_copy(orig_addr, copy)); -} - -static triton_ret_t fault_group_add(const char *service_name, triton_msg_method_group_t *group) -{ - return(base_method->group_add(service_name, group)); -} - -static triton_msg_tag_t fault_new_tag(triton_msg_method_group_t group) -{ - return(base_method->new_tag(group)); -} - - -static __blocking triton_ret_t fault_send( - triton_msg_method_addr_t to, - triton_msg_method_group_t group, - triton_msg_tag_t tag, - int count, - char **buffers, - uint32_t *sizes) -{ - triton_ret_t tret; - int timeouts = 0; - int errors = 0; - int ae_ret; - - if(should_fail(insert_send_timeouts_at_rate)) - { - triton_debug(fault_dbg_mask, "Injecting timeout fault on send.\n"); - ae_ret = aesop_timer(LONG_TIME); - if (ae_ret == AE_SUCCESS) - return(TRITON_SUCCESS); - else - return(TRITON_ERR_UNKNOWN); - } - if(should_fail(insert_send_errors_at_rate)) - { - triton_debug(fault_dbg_mask, "Injecting error fault on send.\n"); - return(TRITON_ERR_MPI); - } - - triton_mutex_lock(&fault_mutex); - ae_ret = aesop_hints_get("triton.net.fault_injector.insert_send_timeouts", - sizeof(timeouts), &timeouts); - if(ae_ret == AE_SUCCESS && timeouts > 0) - { - timeouts--; - aesop_hints_modify("triton.net.fault_injector.insert_send_timeouts", sizeof(timeouts), &timeouts); - triton_debug(fault_dbg_mask, "Injecting timeout fault on send.\n"); - triton_mutex_unlock(&fault_mutex); - ae_ret = aesop_timer(LONG_TIME); - if (ae_ret == AE_SUCCESS) - return(TRITON_SUCCESS); - else - return(TRITON_ERR_UNKNOWN); - } - - ae_ret = aesop_hints_get("triton.net.fault_injector.insert_send_errors", - sizeof(errors), &errors); - if(ae_ret == AE_SUCCESS && errors > 0) - { - errors--; - aesop_hints_modify("triton.net.fault_injector.insert_send_errors", sizeof(errors), &errors); - triton_debug(fault_dbg_mask, "Injecting error fault on send.\n"); - triton_mutex_unlock(&fault_mutex); - return(TRITON_ERR_MPI); - } - - triton_mutex_unlock(&fault_mutex); - - return(base_method->send(to, group, tag, count, buffers, sizes)); -} - -static __blocking triton_ret_t fault_recv( - triton_msg_method_addr_t from, - triton_msg_method_group_t group, - triton_msg_tag_t tag, - int count, - char **buffers, - uint32_t *sizes, - uint32_t *bytes_received) -{ - triton_ret_t tret; - int timeouts = 0; - int errors = 0; - int ae_ret; - - if(should_fail(insert_recv_timeouts_at_rate)) - { - triton_debug(fault_dbg_mask, "Injecting timeout fault on recv.\n"); - ae_ret = aesop_timer(LONG_TIME); - if (ae_ret == AE_SUCCESS) - return(TRITON_SUCCESS); - else - return(TRITON_ERR_UNKNOWN); - } - if(should_fail(insert_recv_errors_at_rate)) - { - triton_debug(fault_dbg_mask, "Injecting error fault on recv.\n"); - return(TRITON_ERR_MPI); - } - - triton_mutex_lock(&fault_mutex); - ae_ret = aesop_hints_get("triton.net.fault_injector.insert_recv_timeouts", - sizeof(timeouts), &timeouts); - if(ae_ret == AE_SUCCESS && timeouts > 0) - { - timeouts--; - aesop_hints_modify("triton.net.fault_injector.insert_recv_timeouts", sizeof(timeouts), &timeouts); - triton_debug(fault_dbg_mask, "Injecting timeout fault on recv.\n"); - triton_mutex_unlock(&fault_mutex); - ae_ret = aesop_timer(LONG_TIME); - if (ae_ret == AE_SUCCESS) - return(TRITON_SUCCESS); - else - return(TRITON_ERR_UNKNOWN); - } - - ae_ret = aesop_hints_get("triton.net.fault_injector.insert_recv_errors", - sizeof(errors), &errors); - if(ae_ret == AE_SUCCESS && errors > 0) - { - errors--; - aesop_hints_modify("triton.net.fault_injector.insert_recv_errors", sizeof(errors), &errors); - triton_debug(fault_dbg_mask, "Injecting error fault on recv.\n"); - triton_mutex_unlock(&fault_mutex); - return(TRITON_ERR_MPI); - } - - triton_mutex_unlock(&fault_mutex); - - - return(base_method->recv(from, group, tag, count, buffers, sizes, bytes_received)); -} - -static __blocking triton_ret_t fault_recv_any( - triton_msg_method_group_t group, - triton_msg_method_addr_t *from, - triton_msg_tag_t *tag, - int count, - char **buffers, - uint32_t *sizes, - uint32_t *bytes_received) -{ - /* NOTE: we deliberately never insert timeout failures in recv_any; by - * definition they can wait indefinitely. - */ - return(base_method->recv_any(group, from, tag, count, buffers, sizes, bytes_received)); -} - - -static triton_ret_t set_one_sided_threshold(size_t size) -{ - return TRITON_ERR_NOT_IMPLEMENTED; -} - -static triton_ret_t get_one_sided_threshold(size_t *size) -{ - return TRITON_ERR_NOT_IMPLEMENTED; -} - -static void* buffer_allocate(size_t size) -{ - return NULL; -} - -static void buffer_free(void* buffer) -{ -} - -/** -* Register memory for the communication subsystem. -*/ -static triton_ret_t register_memory(void* base, //the base address of the memory region - size_t size, // the size in bytes of the memory region - triton_mem_reg_handle_t* handle /*handle returned by - the memory registration*/ - ) -{ - return TRITON_ERR_NOT_IMPLEMENTED; -} - -/** -* Unregister memory. -*/ -static triton_ret_t unregister_memory(triton_mem_reg_handle_t handle /*handle previously returned - by the memory registration*/ - ) -{ - return TRITON_ERR_NOT_IMPLEMENTED; -} - -/** -* Expose a registered memory portion to remote peers. -*/ -static __blocking triton_ret_t publish_memory(triton_mem_reg_handle_t in_handle, //handle of a registered memory - size_t offset, /*offset (in bytes) from the base address - where the memory publishing should start from*/ - size_t size, //size (in bytes) of the published area - triton_mem_pub_handle_t* out_handle /*handle returned from - publishing the memory*/ - ) -{ - return TRITON_ERR_NOT_IMPLEMENTED; -} - - -/** -* End access permissions to a memory portion previously exposed. This is meant to be a client-side operation -*/ -static __blocking triton_ret_t unpublish_memory(triton_mem_pub_handle_t handle /*handle previously - returned by a memory publishing*/ - ) -{ - return TRITON_ERR_NOT_IMPLEMENTED; -} - -static int get_serialized_handle_size(triton_serialized_handle_type_t handle_type, void* handle) -{ - return -1; -} - -static triton_ret_t get_serialized_handle(triton_serialized_handle_type_t handle_type, - void* pointer_to_in_handle, - char* serialized_handle) -{ - return TRITON_ERR_NOT_IMPLEMENTED; -} - -static triton_ret_t create_handle_from_serialization(triton_serialized_handle_type_t handle_type, - void* pointer_to_out_handle, - char* serialized_handle) -{ - return TRITON_ERR_NOT_IMPLEMENTED; -} - -static triton_ret_t free_serialized_handle(triton_serialized_handle_type_t handle_type, - void* pointer_to_in_handle) -{ - return TRITON_ERR_NOT_IMPLEMENTED; -} - -static __blocking triton_ret_t get(triton_msg_method_addr_t from, - triton_mem_pub_handle_t handle, - triton_iov_item_t* iovs, - char** dest_bufs, - int iov_items_count) -{ - return TRITON_ERR_NOT_IMPLEMENTED; -} - -static __blocking triton_ret_t put(triton_msg_method_addr_t to, - triton_mem_pub_handle_t handle, - triton_iov_item_t* iovs, - char** src_bufs, - int iov_items_count) -{ - return TRITON_ERR_NOT_IMPLEMENTED; -} - - -static __blocking triton_ret_t start_epoch(triton_mem_pub_handle_t handle) -{ - return TRITON_ERR_NOT_IMPLEMENTED; -} - - -static __blocking triton_ret_t end_epoch(triton_mem_pub_handle_t handle) -{ - return TRITON_ERR_NOT_IMPLEMENTED; -} - -static __blocking triton_ret_t wait(triton_mem_pub_handle_t mem_handle, - void* wait_info) -{ - return TRITON_ERR_NOT_IMPLEMENTED; -} - -static struct triton_msg_method triton_msg_fault_method = -{ - .name = "fault", - - .addr_self = fault_addr_self, - .addr_equal = fault_addr_equal, - .addr_to_string = fault_addr_to_string, - .addr_lookup = fault_addr_lookup, - .addr_free = fault_addr_free, - .addr_copy = fault_addr_copy, - .new_tag = fault_new_tag, - - .group_add = fault_group_add, - .send = fault_send, - .recv = fault_recv, - .recv_any = fault_recv_any, - - .set_one_sided_threshold = set_one_sided_threshold, - .get_one_sided_threshold = get_one_sided_threshold, - .buffer_allocate = buffer_allocate, - .buffer_free = buffer_free, - .register_memory = register_memory, - .unregister_memory = unregister_memory, - .publish_memory = publish_memory, - .unpublish_memory = unpublish_memory, - .get_serialized_handle_size = get_serialized_handle_size, - .create_handle_from_serialization = create_handle_from_serialization, - .get_serialized_handle = get_serialized_handle, - .free_serialized_handle = free_serialized_handle, - .get = get, - .put = put, - .start_epoch = start_epoch, - .end_epoch = end_epoch, - .wait = wait -}; - -static triton_ret_t base_method_updater(const char *key, const char *value) -{ - char *v; - assert(!strcmp(key, "triton.net.fault_injector.base_method")); - - /* find the actual method string by removing the 'triton.net.' prefix */ - v = strrchr(value, '.'); - if(!v) return TRITON_ERR_INVAL; - if(*v == '\0') return TRITON_ERR_INVAL; - v++; - - base_method = triton_msg_lookup_method(v); - - return TRITON_SUCCESS; -} - -static triton_ret_t insert_faults_at_rate_updater(const char *key, const char *value) -{ - int s; - float frate; - float* target; - - if(!strcmp(key, "triton.net.fault_injector.insert_send_timeouts_at_rate")) - { - target = &insert_send_timeouts_at_rate; - } - else if(!strcmp(key, "triton.net.fault_injector.insert_recv_timeouts_at_rate")) - { - target = &insert_recv_timeouts_at_rate; - } - else if(!strcmp(key, "triton.net.fault_injector.insert_send_errors_at_rate")) - { - target = &insert_send_errors_at_rate; - } - else if(!strcmp(key, "triton.net.fault_injector.insert_recv_errors_at_rate")) - { - target = &insert_recv_errors_at_rate; - } - else - { - return(TRITON_ERR_INVAL); - } - - s = sscanf(value, "%f", &frate); - if(s < 1) - { - return TRITON_ERR_INVAL; - } - - *target = frate; - return TRITON_SUCCESS; -} - -triton_ret_t triton_msg_fault_init(void) -{ - triton_ret_t ret; - struct timeval tv; - unsigned int seed = 0; - int ae_ret = 0; - - triton_mutex_lock(&module_lock); - - if(!module_refcount) - { - ret = triton_zeroconf_init(); - if(ret != TRITON_SUCCESS) - { - triton_mutex_unlock(&module_lock); - return(ret); - } - - ret = triton_debug_init(); - if(ret != TRITON_SUCCESS) - { - triton_mutex_unlock(&module_lock); - triton_zeroconf_finalize(); - return(ret); - } - - ret = triton_msg_init(); - if (ret != TRITON_SUCCESS) - { - triton_mutex_unlock(&module_lock); - triton_debug_finalize(); - triton_zeroconf_finalize(); - return (ret); - } - - ae_ret = ae_hints_component_init(); - if(ae_ret != AE_SUCCESS) - { - triton_mutex_unlock(&module_lock); - triton_debug_finalize(); - triton_zeroconf_finalize(); - triton_msg_finalize(); - return (ret); - } - - ae_ret = aesop_timer_init(); - if(ae_ret != AE_SUCCESS) - { - triton_mutex_unlock(&module_lock); - triton_debug_finalize(); - triton_zeroconf_finalize(); - triton_msg_finalize(); - ae_hints_component_finalize(); - return (ret); - } - - - triton_debug_add_mask("net.fault", &fault_dbg_mask, - "Show debug information for the fault-injector method"); - - /* seed random number generator */ - gettimeofday(&tv, NULL); - seed += tv.tv_usec; - srandom(seed); - - ret = triton_msg_method_register(&triton_msg_fault_method); - triton_error_assert(ret); - - ret = triton_zeroconf_register(base_method_confkey, "triton.net.mpi", base_method_updater, - "Specify the base messaging method for the fault injector"); - triton_error_assert(ret); - - /* zeroconf parameters for persistent fault rates */ - ret = triton_zeroconf_register(insert_send_timeouts_at_rate_confkey, "0", insert_faults_at_rate_updater, - "Specify the rate to insert send timeouts"); - triton_error_assert(ret); - - ret = triton_zeroconf_register(insert_recv_timeouts_at_rate_confkey, "0", insert_faults_at_rate_updater, - "Specify the rate to insert recv timeouts"); - triton_error_assert(ret); - - ret = triton_zeroconf_register(insert_send_errors_at_rate_confkey, "0", insert_faults_at_rate_updater, - "Specify the rate to insert send errors"); - triton_error_assert(ret); - - ret = triton_zeroconf_register(insert_recv_errors_at_rate_confkey, "0", insert_faults_at_rate_updater, - "Specify the rate to insert recv errors"); - triton_error_assert(ret); - - /* hints for injecting faults in specific locations */ - ae_ret = ae_hints_type_register("triton.net.fault_injector.insert_send_timeouts", 0); - assert(ae_ret == AE_SUCCESS); - ae_ret = ae_hints_type_register("triton.net.fault_injector.insert_recv_timeouts", 0); - assert(ae_ret == AE_SUCCESS); - ae_ret = ae_hints_type_register("triton.net.fault_injector.insert_send_errors", 0); - assert(ae_ret == AE_SUCCESS); - ae_ret = ae_hints_type_register("triton.net.fault_injector.insert_recv_errors", 0); - assert(ae_ret == AE_SUCCESS); - } - module_refcount++; - triton_mutex_unlock(&module_lock); - - return ret; -} - -void triton_msg_fault_finalize(void) -{ - triton_mutex_lock(&module_lock); - module_refcount--; - - if(!module_refcount) - { - triton_msg_method_unregister("fault"); - triton_debug_finalize(); - triton_zeroconf_finalize(); - triton_msg_finalize(); - ae_hints_component_finalize(); - aesop_timer_finalize(); - } - triton_mutex_unlock(&module_lock); -} - diff --git a/code/src/net/fault-injector/fault-method.h b/code/src/net/fault-injector/fault-method.h deleted file mode 100644 index 79ed523..0000000 --- a/code/src/net/fault-injector/fault-method.h +++ /dev/null @@ -1,9 +0,0 @@ -#ifndef __FAULT_METHOD_H__ -#define __FAULT_METHOD_H__ - -#include "src/common/triton-error.h" - -triton_ret_t triton_msg_fault_init(void); -void triton_msg_fault_finalize(void); - -#endif /* __FAULT_METHOD_H__ */ diff --git a/code/src/net/fault-injector/module.mk.in b/code/src/net/fault-injector/module.mk.in deleted file mode 100644 index 5271073..0000000 --- a/code/src/net/fault-injector/module.mk.in +++ /dev/null @@ -1,7 +0,0 @@ -DIR := src/net/fault-injector - -ifneq (,$(BUILD_MPI)) - -AELIBSRC += $(DIR)/fault-method.ae - -endif # BUILD_MPI diff --git a/code/src/net/module.mk.in b/code/src/net/module.mk.in deleted file mode 100644 index ba3c9c9..0000000 --- a/code/src/net/module.mk.in +++ /dev/null @@ -1,7 +0,0 @@ -DIR := src/net - -AESOP_HDR += $(DIR)/triton-message.hae \ - $(DIR)/triton-message-method.hae - -AELIBSRC += $(DIR)/triton-message.ae - diff --git a/code/src/net/mpi/module.mk.in b/code/src/net/mpi/module.mk.in deleted file mode 100644 index 4ec911e..0000000 --- a/code/src/net/mpi/module.mk.in +++ /dev/null @@ -1,10 +0,0 @@ -DIR := src/net/mpi - -ifneq (,$(BUILD_MPI)) - -AELIBSRC += $(DIR)/mpi-method.ae - -MODCFLAGS_$(DIR)/mpi-method = $(MPICFLAGS) -MODCC_$(DIR)/mpi-method = $(MPICC) - -endif # BUILD_MPI diff --git a/code/src/net/mpi/mpi-method.ae b/code/src/net/mpi/mpi-method.ae deleted file mode 100644 index 141a74b..0000000 --- a/code/src/net/mpi/mpi-method.ae +++ /dev/null @@ -1,1943 +0,0 @@ -/* - * (C) 2009 The University of Chicago - * - * See COPYING in top-level directory. - */ - -#include <unistd.h> -#include <mpi.h> - -#include <triton-hash.h> -#include <aesop/sem.hae> - -#include "src/net/mpi/mpi-method.h" -#include "src/net/triton-addr.h" -#include "src/net/triton-message.hae" -#include "src/net/triton-message-method.hae" -#include "src/common/triton-log.h" -#include "src/remote/byteswap.h" - -static triton_mutex_t module_lock = TRITON_MUTEX_INITIALIZER; -static int module_refcount = 0; - -#define PRINT_ERR_MSG(err_msg_format,...)\ -do {\ - triton_err(triton_log_default, "%s:%d: [MPI Proc rank %d]" err_msg_format,\ - __FILE__, __LINE__, (int)mpi_rank, ##__VA_ARGS__);\ -}while(0) - -#define ABORT(err_msg_format,...)\ -do { \ - PRINT_ERR_MSG(err_msg_format,##__VA_ARGS__); \ - MPI_Abort(MPI_COMM_WORLD, ret); \ -}while(0) - - -#define ABORT_IF(cond,err_msg_format,...) do{if((cond)) ABORT(err_msg_format,##__VA_ARGS__);}while(0) - -#define ABORT_ON_MPI_FAILURE(ret,err_msg_format,...) ABORT_IF((ret)!=MPI_SUCCESS,err_msg_format,##__VA_ARGS__) - -#define RETURN_ON_MPI_FAILURE(mpi_ret_code,err_msg_format,...)\ -do { \ - if((mpi_ret_code) != MPI_SUCCESS) \ - { \ - PRINT_ERR_MSG(err_msg_format,##__VA_ARGS__); \ - return triton_error_from_mpi((mpi_ret_code)); \ - } \ -}while(0) - -#define KB 1024 -#define MB 1048576 -#define TRITON_DEFAULT_MPI_MAX_USABLE_REGISTERED_MEM_SIZE (4*MB) - -#define ZERO_SIZE_MEM_HANDLE 0x80000000CAFEBABE - -#define WORD_SIZE SIZEOF_VOID_P - -#define DEFAULT_MPI_ONE_SIDED_SIZE_THRESHOLD 0 /* this is - temporarily set to - 0 for tests - purposes should be - changed to some - sane value (4K?) */ - -static uint64_t mpi_one_sided_size_threshold = - DEFAULT_MPI_ONE_SIDED_SIZE_THRESHOLD; -static uint32_t mpi_internal_one_sided_buffer_size = - TRITON_DEFAULT_MPI_MAX_USABLE_REGISTERED_MEM_SIZE; - -#define ENV_INTERNAL_ONE_SIDED_BUFFER_SIZE "INTERNAL_ONE_SIDED_BUF_SIZE" - - -static triton_msg_method_addr_t mpi_rank; - -static struct triton_hash_table *group_table = NULL; - -struct nb_request -{ - MPI_Request request; - aesop_sem_t sem; - triton_ret_t tret; - int bytes_received; - MPI_Datatype dtype; - int from; - int tag; - triton_list_link_t link; -}; -TRITON_LIST_DEFINE(posted_queue); -TRITON_LIST_DEFINE(inflight_queue); -TRITON_LIST_DEFINE(completed_queue); - -static triton_mutex_t thread_mutex = TRITON_MUTEX_INITIALIZER; -static pthread_t thread_id; -static int thread_stop_flag = 0; -static int thread_wake_flag = 0; -static int mpi_already_up = 0; -static int mpi_did_init = 0; - -static void* thread_fn(void* arg); -static void wake_mpi_thread(void); -static void internal_finalize(void); -static triton_ret_t triton_error_from_mpi(int r); - -static __blocking triton_msg_method_addr_t mpi_addr_self( - void) -{ - return mpi_rank; -} - -static int mpi_addr_equal( - triton_msg_method_addr_t a1, - triton_msg_method_addr_t a2) -{ - return a1 == a2; -} - -static triton_ret_t mpi_addr_to_string( - triton_msg_method_addr_t addr, - triton_string_t * addrstr) -{ - triton_string_init(addrstr, "%u", (uint32_t) addr); - return TRITON_SUCCESS; -} - -static triton_ret_t mpi_addr_free( - triton_msg_method_addr_t addr) -{ - // Nothing to do - return TRITON_SUCCESS; -} - -static triton_ret_t mpi_addr_copy( - const triton_msg_method_addr_t orig_addr, - triton_msg_method_addr_t * copy) -{ - *copy = orig_addr; - return TRITON_SUCCESS; -} - -static triton_ret_t mpi_addr_lookup( - const char *name, - triton_msg_method_addr_t * addr) -{ - *addr = atoi(name); - return TRITON_SUCCESS; -} - -struct triton_mpi_group -{ - char *name; - triton_msg_method_group_t group; - MPI_Comm comm; - int tag_counter; - triton_mutex_t mutex; - struct triton_hash_link link; -}; - -static triton_ret_t mpi_group_add( - const char *service_name, - triton_msg_method_group_t * group) -{ - int ret; - uint32_t h1 = 0, h2 = 0; - struct triton_mpi_group *newgroup; - - assert(service_name); - assert(group); - - newgroup = malloc(sizeof(*newgroup)); - if (newgroup == NULL) - return TRITON_ERR_NOMEM; - - newgroup->name = strdup(service_name); - - bj_hashlittle2(service_name, strlen(service_name), &h1, &h2); - newgroup->group = h1; - newgroup->tag_counter = 0; - triton_mutex_init(&newgroup->mutex, NULL); - *group = newgroup->group; - - ret = MPI_Comm_dup(MPI_COMM_WORLD, &newgroup->comm); - if (ret != MPI_SUCCESS) - { - return triton_error_from_mpi(ret); - } - - triton_list_link_clear(&newgroup->link); - triton_hash_add(group_table, &newgroup->group, &newgroup->link); - return TRITON_SUCCESS; -} - -static triton_msg_tag_t mpi_new_tag( - triton_msg_method_group_t group) -{ - struct triton_hash_link *groupl; - struct triton_mpi_group *group_entry; - triton_msg_tag_t tag; - - /* lookup communicator */ - groupl = triton_hash_search(group_table, &group); - group_entry = triton_hash_get_entry(groupl, struct triton_mpi_group, - link); - triton_mutex_lock(&group_entry->mutex); - ++(group_entry->tag_counter); - if (group_entry->tag_counter >= MPI_TAG_UB) - { - group_entry->tag_counter = 0; - } - tag = group_entry->tag_counter; - triton_mutex_unlock(&group_entry->mutex); - return tag; -} - - -static __blocking triton_ret_t mpi_send( - triton_msg_method_addr_t to, - triton_msg_method_group_t group, - triton_msg_tag_t tag, - int count, - char **buffers, - uint32_t * sizes) -{ - int ret, i; - MPI_Datatype hind_type; - MPI_Aint *offsets; - struct triton_hash_link *groupl; - struct triton_mpi_group *group_entry; - struct nb_request* nb_req; - triton_ret_t tret; - - offsets = malloc(sizeof(*offsets) * count); - if (offsets == NULL) - { - return TRITON_ERR_NOMEM; - } - - nb_req = malloc(sizeof(*nb_req)); - if(!nb_req) - { - free(offsets); - return TRITON_ERR_NOMEM; - } - triton_list_link_clear(&nb_req->link); - /* TODO: do I have to call sem_destroy on every error path now? */ - aesop_sem_init(&nb_req->sem, 0); - - for (i = 0; i < count; ++i) - { - MPI_Get_address(buffers[i], &offsets[i]); - } - - ret = - MPI_Type_create_hindexed(count, (int *) sizes, offsets, MPI_BYTE, - &hind_type); - if (ret != MPI_SUCCESS) - { - free(nb_req); - free(offsets); - return triton_error_from_mpi(ret); - } - - ret = MPI_Type_commit(&hind_type); - if (ret != MPI_SUCCESS) - { - free(nb_req); - free(offsets); - MPI_Type_free(&hind_type); - return triton_error_from_mpi(ret); - } - - /* lookup communicator */ - groupl = triton_hash_search(group_table, &group); - group_entry = triton_hash_get_entry(groupl, struct triton_mpi_group, link); - - nb_req->bytes_received = 0; - nb_req->dtype = hind_type; - ret = MPI_Isend(MPI_BOTTOM, 1, hind_type, (int)to, tag, group_entry->comm, - &nb_req->request); - if(ret != MPI_SUCCESS) - { - free(nb_req); - free(offsets); - MPI_Type_free(&hind_type); - return(triton_error_from_mpi(ret)); - } - - triton_mutex_lock(&thread_mutex); - triton_list_add_back(&nb_req->link, &posted_queue); - triton_mutex_unlock(&thread_mutex); - wake_mpi_thread(); - - ret = aesop_sem_down(&nb_req->sem); - - triton_mutex_lock(&thread_mutex); - if(ret == 0 || (triton_list_return_list(&nb_req->link) == &completed_queue)) - { - /* operation completed normally, or was cancelled after already - * completed - */ - assert(triton_list_return_list(&nb_req->link) == &completed_queue); - - triton_list_del(&nb_req->link); - tret = nb_req->tret; - aesop_sem_destroy(&nb_req->sem); - free(nb_req); - } - else - { - /* operation cancelled */ - /* It could be on the posted_queue or on the inflight_queue at this - * point. Either way we still have to call MPI_Cancel() and allow - * MPI_Waitsome() to complete the operation. - * - * We clear the aesop cancel flag here to allow a second - * sem_down() call to block until the cancellation is complete. We - * do not expect this block of code to be cancelled again. - */ - MPI_Cancel(&nb_req->request); - triton_mutex_unlock(&thread_mutex); - - aesop_clear_cancel(); - ret = aesop_sem_down(&nb_req->sem); - assert(ret == 0); - - triton_mutex_lock(&thread_mutex); - - triton_list_del(&nb_req->link); - tret = TRITON_ERR_CANCELED; - aesop_sem_destroy(&nb_req->sem); - free(nb_req); - } - triton_mutex_unlock(&thread_mutex); - - free(offsets); - return(tret); -} - -static __blocking triton_ret_t mpi_recv( - triton_msg_method_addr_t from, - triton_msg_method_group_t group, - triton_msg_tag_t tag, - int count, - char **buffers, - uint32_t * sizes, - uint32_t * bytes_received) -{ - int ret, i; - MPI_Datatype hind_type; - MPI_Aint *offsets; - struct triton_hash_link *groupl; - struct triton_mpi_group *group_entry; - MPI_Status status; - triton_ret_t tret; - struct nb_request *nb_req; - - offsets = malloc(sizeof(*offsets) * count); - if (offsets == NULL) - { - return TRITON_ERR_NOMEM; - } - - nb_req = malloc(sizeof(*nb_req)); - if(!nb_req) - { - free(offsets); - return TRITON_ERR_NOMEM; - } - triton_list_link_clear(&nb_req->link); - /* TODO: do I have to call sem_destroy on every error path now? */ - aesop_sem_init(&nb_req->sem, 0); - - for (i = 0; i < count; ++i) - { - MPI_Get_address(buffers[i], &offsets[i]); - } - - ret = - MPI_Type_create_hindexed(count, (int *) sizes, offsets, MPI_BYTE, - &hind_type); - if (ret != MPI_SUCCESS) - { - free(nb_req); - free(offsets); - return triton_error_from_mpi(ret); - } - - ret = MPI_Type_commit(&hind_type); - if (ret != MPI_SUCCESS) - { - free(nb_req); - free(offsets); - MPI_Type_free(&hind_type); - return triton_error_from_mpi(ret); - } - - /* lookup communicator */ - groupl = triton_hash_search(group_table, &group); - group_entry = triton_hash_get_entry(groupl, struct triton_mpi_group, link); - - nb_req->bytes_received = 1; - nb_req->dtype = hind_type; - ret = MPI_Irecv(MPI_BOTTOM, 1, hind_type, (int)from, tag, group_entry->comm, - &nb_req->request); - if(ret != MPI_SUCCESS) - { - free(nb_req); - free(offsets); - MPI_Type_free(&hind_type); - return(triton_error_from_mpi(ret)); - } - - triton_mutex_lock(&thread_mutex); - triton_list_add_back(&nb_req->link, &posted_queue); - triton_mutex_unlock(&thread_mutex); - wake_mpi_thread(); - - ret = aesop_sem_down(&nb_req->sem); - - triton_mutex_lock(&thread_mutex); - if(ret == 0) - { - /* operation completed normally */ - assert(triton_list_return_list(&nb_req->link) == &completed_queue); - - triton_list_del(&nb_req->link); - tret = nb_req->tret; - *bytes_received = nb_req->bytes_received; - aesop_sem_destroy(&nb_req->sem); - free(nb_req); - } - else - { - /* operation cancelled */ - /* It could be on the posted_queue or on the inflight_queue at this - * point. Either way we still have to call MPI_Cancel() and allow - * MPI_Waitsome() to complete the operation. - * - * We clear the aesop cancel flag here to allow a second - * sem_down() call to block until the cancellation is complete. We - * do not expect this block of code to be cancelled again. - */ - MPI_Cancel(&nb_req->request); - triton_mutex_unlock(&thread_mutex); - - aesop_clear_cancel(); - ret = aesop_sem_down(&nb_req->sem); - assert(ret == 0); - - triton_mutex_lock(&thread_mutex); - - triton_list_del(&nb_req->link); - tret = TRITON_ERR_CANCELED; - aesop_sem_destroy(&nb_req->sem); - free(nb_req); - } - triton_mutex_unlock(&thread_mutex); - - free(offsets); - return(tret); -} - -static __blocking triton_ret_t mpi_recv_any( - triton_msg_method_group_t group, - triton_msg_method_addr_t * from, - triton_msg_tag_t * tag, - int count, - char **buffers, - uint32_t * sizes, - uint32_t * bytes_received) -{ - int ret, i; - MPI_Datatype hind_type; - MPI_Aint *offsets; - struct triton_hash_link *groupl; - struct triton_mpi_group *group_entry; - MPI_Status status; - struct nb_request* nb_req; - triton_ret_t tret; - - offsets = malloc(sizeof(*offsets) * count); - if (offsets == NULL) - { - return TRITON_ERR_NOMEM; - } - - nb_req = malloc(sizeof(*nb_req)); - if(!nb_req) - { - free(offsets); - return TRITON_ERR_NOMEM; - } - triton_list_link_clear(&nb_req->link); - /* TODO: do I have to call sem_destroy on every error path now? */ - aesop_sem_init(&nb_req->sem, 0); - - for (i = 0; i < count; ++i) - { - MPI_Get_address(buffers[i], &offsets[i]); - } - - ret = - MPI_Type_create_hindexed(count, (int *) sizes, offsets, MPI_BYTE, - &hind_type); - if (ret != MPI_SUCCESS) - { - free(nb_req); - free(offsets); - return triton_error_from_mpi(ret); - } - - ret = MPI_Type_commit(&hind_type); - if (ret != MPI_SUCCESS) - { - free(nb_req); - free(offsets); - MPI_Type_free(&hind_type); - return triton_error_from_mpi(ret); - } - - /* lookup communicator */ - groupl = triton_hash_search(group_table, &group); - group_entry = triton_hash_get_entry(groupl, struct triton_mpi_group, link); - - nb_req->bytes_received = 1; - nb_req->dtype = hind_type; - ret = MPI_Irecv(MPI_BOTTOM, 1, hind_type, MPI_ANY_SOURCE, MPI_ANY_TAG, group_entry->comm, - &nb_req->request); - if(ret != MPI_SUCCESS) - { - free(nb_req); - free(offsets); - MPI_Type_free(&hind_type); - return(triton_error_from_mpi(ret)); - } - - triton_mutex_lock(&thread_mutex); - triton_list_add_back(&nb_req->link, &posted_queue); - triton_mutex_unlock(&thread_mutex); - wake_mpi_thread(); - - ret = aesop_sem_down(&nb_req->sem); - - triton_mutex_lock(&thread_mutex); - if(ret == 0) - { - /* operation completed normally */ - assert(triton_list_return_list(&nb_req->link) == &completed_queue); - - triton_list_del(&nb_req->link); - tret = nb_req->tret; - *bytes_received = nb_req->bytes_received; - *from = nb_req->from; - *tag = nb_req->tag; - aesop_sem_destroy(&nb_req->sem); - free(nb_req); - } - else - { - /* operation cancelled */ - /* It could be on the posted_queue or on the inflight_queue at this - * point. Either way we still have to call MPI_Cancel() and allow - * MPI_Waitsome() to complete the operation. - * - * We clear the aesop cancel flag here to allow a second - * sem_down() call to block until the cancellation is complete. We - * do not expect this block of code to be cancelled again. - */ - MPI_Cancel(&nb_req->request); - triton_mutex_unlock(&thread_mutex); - - aesop_clear_cancel(); - ret = aesop_sem_down(&nb_req->sem); - assert(ret == 0); - - triton_mutex_lock(&thread_mutex); - - triton_list_del(&nb_req->link); - tret = TRITON_ERR_CANCELED; - aesop_sem_destroy(&nb_req->sem); - free(nb_req); - } - triton_mutex_unlock(&thread_mutex); - - free(offsets); - return(tret); -} - -static int mpi_group_compare( - const void *key, - struct triton_hash_link *h_link) -{ - const triton_msg_method_group_t *group1 = - (const triton_msg_method_group_t *) key; - struct triton_mpi_group *mpi_group = - triton_hash_get_entry(h_link, struct triton_mpi_group, link); - assert(key); - assert(h_link); - - return (mpi_group->group == *group1); -} - -static void group_entry_free( - void *x) -{ - struct triton_mpi_group *g; - - g = (struct triton_mpi_group *) x; - - triton_mutex_destroy(&g->mutex); - MPI_Comm_free(&g->comm); - free(g->name); - free(g); -} - -/*---------------One-sided --------------------*/ - -//TODO: Fix linked lists to use the triton ones - -typedef struct node_link_t -{ - struct node_link_t *next; -} node_link_t; - -static void add_node( - void **node_head, - void **node_tail, - void *node) -{ - node_link_t **head = (node_link_t **) (node_head); - node_link_t **tail = (node_link_t **) (node_tail); - if (*tail) - { - (*tail)->next = (node_link_t *) node; - } - else - { - *head = (node_link_t *) node; - } - *tail = (node_link_t *) node; -} - -static void remove_node( - void **node_head, - void **node_tail, - void *node) -{ - node_link_t *cur = (node_link_t *) (*node_head), *previous = NULL; - while (cur) - { - if (cur == node) - { - if (previous) - previous->next = cur->next; - else - *node_head = cur->next; - if (*node_tail == cur) - *node_tail = previous; - break; - } - previous = cur; - cur = cur->next; - } -} - -static triton_ret_t set_one_sided_threshold( - size_t size) -{ - // triton_assert((int64_t)size >= 0); - mpi_one_sided_size_threshold = size; - return TRITON_SUCCESS; -} - - -static triton_ret_t get_one_sided_threshold( - size_t * size) -{ - *size = (size_t) mpi_one_sided_size_threshold; - return TRITON_SUCCESS; -} - - -typedef struct mem_reg_t -{ - void *base_address; // address of the portion taken from the - // buffer exposed for one-sided - // this should be normally size long - void *user_address; /* address provided by the user. This pointer - is the same as base_address if the handle - has been created through buffer_allocate - We use the following convention: 1- - base_address == user_address => reg_handle - is associated with a buffer_allocated - memory and that memory has been registered - as well 2- user_address == NULL => - reg_handle is associated with a - buffer_allocated memory and that memory - has not been registered or is already - unregistered 3- base_address != - user_address && user_address != NULL => - reg_handle is associated with a - non-buffer_allocated memory that is - currently registered */ - uint64_t user_size; // user_size is the size provided by the user - // when registering - uint64_t size; /* is the actual size used internally. - size==user_size if user_size%WORD_SIZE is - 0 otherwise it is - ceil(1.0*user_size/WORD_SIZE)*WORD_SIZE */ - int publish_count; -} mem_reg_t; - -typedef struct mem_range_t -{ - struct mem_range_t *next; - mem_reg_t *mem_reg_info; - uint64_t offset; - uint64_t size; - int made_from_deserialization; // 1 if made from deserialization - int client_rank; // the rank of the peer that initially - // published the memory -} mem_range_t; - -typedef mem_range_t mem_pub_t; - -typedef struct locked_rank_t -{ - struct locked_rank_t *next; - int rank; - int lock_count; // how many pending lock request for this - // remote rank. The count includes the - // currently holding - triton_mutex_t mutex; // because the same rank cannot be locked - // more than once by the same process -} locked_rank_t; - -struct -{ - int rank; - int size; - MPI_Win *wins; // the window handles - - void *one_sided_memory; // It is the memory registered for one-sided - uint64_t remaining_mem_size; // remaining usable size in - // one_sided_memory - triton_mutex_t mutex; - mem_range_t *mem_range_head; // Head of memory slices. - locked_rank_t *locked_rank_head; - locked_rank_t *locked_rank_tail; - -} mpi_global_data = -{ --1, -1, NULL, NULL, 0, TRITON_MUTEX_INITIALIZER, NULL, NULL, NULL}; - - -static locked_rank_t *find_locked_rank( - int rank) -{ - locked_rank_t *lr = mpi_global_data.locked_rank_head; - while (lr) - { - if (lr->rank == rank) - return lr; - lr = lr->next; - } - return NULL; -} - -static inline int is_rank_locked( - int rank) -{ - return find_locked_rank(rank) != NULL; -} - -static inline void remove_locked_rank( - int rank) -{ - locked_rank_t *lr = find_locked_rank(rank); - if (lr) - { - remove_node((void **) (&mpi_global_data.locked_rank_head), - (void **) (&mpi_global_data.locked_rank_tail), (void *) lr); - triton_mutex_destroy(&lr->mutex); - free(lr); - } -} - -static locked_rank_t *add_locked_rank( - int rank) -{ - locked_rank_t *lr = malloc(sizeof(locked_rank_t)); - lr->next = NULL; - lr->rank = rank; - lr->lock_count = 0; - triton_mutex_init(&lr->mutex, NULL); - add_node((void **) (&mpi_global_data.locked_rank_head), - (void **) (&mpi_global_data.locked_rank_tail), (void *) lr); - return lr; -} - - -/*Find the mem_range_t* associated with a certain address -*/ -static mem_range_t *find_associated_mem_range( - void *address) -{ - mem_range_t *cur_mem_range_handle = NULL; - for (cur_mem_range_handle = mpi_global_data.mem_range_head; - cur_mem_range_handle; - cur_mem_range_handle = cur_mem_range_handle->next) - { - if (cur_mem_range_handle->mem_reg_info && - cur_mem_range_handle->mem_reg_info->base_address == address) - return cur_mem_range_handle; - } - return NULL; -} - - -static triton_ret_t register_memory_internal( - void *base, - uint64_t size, - mem_reg_t ** handle) -{ - mem_range_t *cur_mem_range_handle = NULL; - mem_range_t *prev_mem_range_handle = NULL; - mem_range_t *found_mem_range = NULL; - mem_reg_t *mem_reg_info = NULL; - *handle = NULL; - uint64_t user_size = size; - - if ((size % WORD_SIZE) != 0) - size = (size + WORD_SIZE) - (size % WORD_SIZE); - /* Base might have been allocated with buffer_allocate in which case we - only need to return its associated handle */ - mem_range_t *range = find_associated_mem_range(base); - if (range) - { - // (See the comment on mem_reg_t to understand the instruction right - // below) - range->mem_reg_info->user_address = range->mem_reg_info->base_address; - *handle = range->mem_reg_info; - return TRITON_SUCCESS; - } - - // Check for space availability - if (size > mpi_global_data.remaining_mem_size) - { - return TRITON_ERR_NOMEM; - } - - triton_mutex_lock(&mpi_global_data.mutex); - - // check if any contiguous memory slice exists that is at least size - cur_mem_range_handle = mpi_global_data.mem_range_head; - while (cur_mem_range_handle) - { - if (cur_mem_range_handle->mem_reg_info == NULL) // the range is not - // currently - // registered - { - if (cur_mem_range_handle->size > size) - { - /* slice some memory for the registration */ - - mem_range_t *new_mem_range_handle = - (mem_range_t *) malloc(sizeof(mem_range_t)); - if (new_mem_range_handle == NULL) - { - triton_mutex_unlock(&mpi_global_data.mutex); - return TRITON_ERR_NOMEM; - } - mem_reg_info = (mem_reg_t *) malloc(sizeof(mem_reg_t)); - if (mem_reg_info == NULL) - { - triton_mutex_unlock(&mpi_global_data.mutex); - return TRITON_ERR_NOMEM; - } - mem_reg_info->publish_count = 0; - - new_mem_range_handle->offset = cur_mem_range_handle->offset; - new_mem_range_handle->size = size; - - cur_mem_range_handle->offset += size; - cur_mem_range_handle->size -= size; - - if (prev_mem_range_handle) - prev_mem_range_handle->next = new_mem_range_handle; - else - mpi_global_data.mem_range_head = new_mem_range_handle; - new_mem_range_handle->next = cur_mem_range_handle; - - found_mem_range = new_mem_range_handle; - break; - } - else if (cur_mem_range_handle->size == size) - { - found_mem_range = cur_mem_range_handle; - mem_reg_info = malloc(sizeof(mem_reg_t)); - if (mem_reg_info == NULL) - { - triton_mutex_unlock(&mpi_global_data.mutex); - return TRITON_ERR_NOMEM; - } - mem_reg_info->publish_count = 0; - break; - } - } - prev_mem_range_handle = cur_mem_range_handle; - cur_mem_range_handle = cur_mem_range_handle->next; - } - - if (found_mem_range != NULL) - { - mpi_global_data.remaining_mem_size -= size; - found_mem_range->mem_reg_info = mem_reg_info; - mem_reg_info->base_address = - (void *) ((uint64_t) mpi_global_data.one_sided_memory + - found_mem_range->offset); - mem_reg_info->user_address = base; /* base is NULL if this - function is called from - buffer_allocate. */ - mem_reg_info->user_size = user_size; - mem_reg_info->size = size; - *handle = mem_reg_info; - triton_mutex_unlock(&mpi_global_data.mutex); - return TRITON_SUCCESS; - } - triton_mutex_unlock(&mpi_global_data.mutex); - return TRITON_ERR_NOMEM; -} - -//Just as the routine name says ... -static void compact_free_contiguous_mem_ranges( - ) -{ - mem_range_t *cur_mem_range_handle = mpi_global_data.mem_range_head; - mem_range_t *prev_mem_range_handle = NULL; - mem_range_t *temp_mem_range_handle = NULL; - - while (cur_mem_range_handle) - { - if (prev_mem_range_handle && prev_mem_range_handle->mem_reg_info == NULL - && cur_mem_range_handle->mem_reg_info == NULL) - { - prev_mem_range_handle->next = cur_mem_range_handle->next; - prev_mem_range_handle->size += cur_mem_range_handle->size; - temp_mem_range_handle = cur_mem_range_handle; - cur_mem_range_handle = cur_mem_range_handle->next; - free(temp_mem_range_handle); - continue; - } - prev_mem_range_handle = cur_mem_range_handle; - cur_mem_range_handle = cur_mem_range_handle->next; - } -} - -static triton_ret_t unregister_memory_internal( - mem_reg_t * mem_reg_info) -{ - mem_range_t *cur_mem_range_handle = mpi_global_data.mem_range_head; - - if (!mem_reg_info) - return TRITON_ERR_INVAL; - - if (mem_reg_info->publish_count != 0) - return TRITON_ERR_MEM_IN_USE; - - triton_mutex_lock(&mpi_global_data.mutex); - if (mem_reg_info->base_address == mem_reg_info->user_address) - { - /* The handle is asociated with an address created through - buffer_allocate. We can't destroy it until buffer_free is called - on the associated address. Though, we have to make it impossible - to call unregister on it again. (See the comment on mem_reg_t to - understand the comment above) */ - mem_reg_info->user_address = NULL; - triton_mutex_unlock(&mpi_global_data.mutex); - return TRITON_SUCCESS; - } - while (cur_mem_range_handle) - { - if (cur_mem_range_handle->mem_reg_info == mem_reg_info) - { - mpi_global_data.remaining_mem_size += mem_reg_info->size; - free(cur_mem_range_handle->mem_reg_info); - cur_mem_range_handle->mem_reg_info = NULL; - compact_free_contiguous_mem_ranges(); - triton_mutex_unlock(&mpi_global_data.mutex); - return TRITON_SUCCESS; - } - cur_mem_range_handle = cur_mem_range_handle->next; - } - triton_mutex_unlock(&mpi_global_data.mutex); - - return TRITON_ERR_INVAL; // The handle is not a valid registered - // memory handle. -} - -static void *buffer_allocate( - size_t size) -{ - mem_reg_t *handle = NULL; - triton_ret_t ret = register_memory_internal(NULL, (uint64_t) size, &handle); - if (ret != TRITON_SUCCESS) - return NULL; - return handle->base_address; -} - -static void buffer_free( - void *buffer) -{ - mem_range_t *range = find_associated_mem_range(buffer); - triton_ret_t ret; - - triton_assert(range); - if (range) - { - if (range->mem_reg_info->base_address == - range->mem_reg_info->user_address) - { - triton_err(triton_log_default, - "%s:%d: Attempt to free a memory that has not yet been unpublished", - __FILE__, __LINE__); - return; - } - ret = unregister_memory_internal(range->mem_reg_info); - triton_assert(ret == TRITON_SUCCESS); - } - else - { - triton_assert(0); - /* trying to free a memory region that has not been allocated with - this mechanism! Shouldn't we trigger a kind of cataclismic event - ... just like the stdlib.h free would do. In debug mode, the - asssert will trigger; but in release mode, if we're silent, the - programmer might think that everything is ok while he is leaking - memory (buffer will not be duly freed through the appropriate - mechanism if the programmer thinks that THIS mechanism is already - doing it. */ - } -} - -/** -* Register memory for the communication subsystem. -*/ -static triton_ret_t register_memory( - void *base, // the base address of the memory region - size_t size, // the size in bytes of the memory region - triton_mem_reg_handle_t * handle /* handle returned by the memory - registration */ - ) -{ - if (size == 0) - { - *handle = ZERO_SIZE_MEM_HANDLE; - return TRITON_SUCCESS; - } - return register_memory_internal(base, (uint64_t) size, - (mem_reg_t **) (handle)); -} - -/** -* Unregister memory. -*/ -static triton_ret_t unregister_memory( - triton_mem_reg_handle_t handle /* handle previously returned by the - memory registration */ - ) -{ - if (handle == ZERO_SIZE_MEM_HANDLE) - return TRITON_SUCCESS; - return unregister_memory_internal((mem_reg_t *) (((void *) (handle)))); -} - -static int is_valid_mem_reg_info( - mem_reg_t * mem_reg_info) -{ - mem_range_t *cur_mem_range_handle = mpi_global_data.mem_range_head; - - while (cur_mem_range_handle) - { - if (cur_mem_range_handle->mem_reg_info == mem_reg_info) - return 1; - cur_mem_range_handle = cur_mem_range_handle->next; - } - return 0; -} - -/** -* Expose a registered memory portion to remote peers. -*/ -static __blocking triton_ret_t publish_memory( - triton_mem_reg_handle_t in_handle, // handle of a registered memory - size_t offset, /* offset (in bytes) from the base address - where the memory publishing should start - from */ - size_t size, // size (in bytes) of the published area - triton_mem_pub_handle_t * out_handle /* handle returned from - publishing the memory */ - ) -{ - if (size == 0) - { - *out_handle = ZERO_SIZE_MEM_HANDLE; - return TRITON_SUCCESS; - } - - mem_reg_t *mem_reg_info = (mem_reg_t *) in_handle; - mem_pub_t *mem_pub_info = NULL; - - if (!is_valid_mem_reg_info(mem_reg_info)) - return TRITON_ERR_INVAL; - - if (mem_reg_info->user_size < (uint64_t) offset || - (uint64_t) size > mem_reg_info->user_size - (uint64_t) offset) - return TRITON_ERR_MEM_OUT_OF_RANGE; - - mem_pub_info = (mem_pub_t *) calloc(1, sizeof(mem_pub_t)); - if (!mem_pub_info) - return TRITON_ERR_NOMEM; - - mem_pub_info->offset = offset + ((uint64_t) mem_reg_info->base_address - - (uint64_t) - mpi_global_data.one_sided_memory); - mem_pub_info->size = size; - mem_pub_info->mem_reg_info = mem_reg_info; - mem_pub_info->client_rank = mpi_global_data.rank; - - triton_mutex_lock(&mpi_global_data.mutex); - mem_reg_info->publish_count++; - triton_mutex_unlock(&mpi_global_data.mutex); - - // if the user-address has not been allocated using buffer_allocate - if (mem_reg_info->base_address != mem_reg_info->user_address) - { - /* The user data must be copied to the mapped one-sided memory so - that any one-sided seamingly reading from the user memory could - effectively grab data put by the user. */ - memcpy((void *) ((uint64_t) mpi_global_data.one_sided_memory + - mem_pub_info->offset), - (void *) ((uint64_t) mem_reg_info->user_address + - mem_pub_info->offset), mem_pub_info->size); - } - *out_handle = (uint64_t) mem_pub_info; - - return TRITON_SUCCESS; -} - -/** -* End access permissions to a memory portion previously exposed. -*/ -static __blocking triton_ret_t unpublish_memory( - triton_mem_pub_handle_t handle /* handle previously returned by a - memory publishing */ - ) -{ - if (handle == ZERO_SIZE_MEM_HANDLE) - return TRITON_SUCCESS; - - mem_pub_t *mem_pub_info = (mem_pub_t *) handle; - mem_reg_t *mem_reg_info = mem_pub_info->mem_reg_info; - - if (!is_valid_mem_reg_info(mem_reg_info)) - return TRITON_ERR_INVAL; - if (mem_pub_info->made_from_deserialization) - return TRITON_ERR_INVAL; // this handle is not allowed to - // unpublish stuff - - // if the user-address has not been allocated using buffer_allocate - if (mem_reg_info->base_address != mem_reg_info->user_address) - { - memcpy((void *) ((uint64_t) mem_reg_info->user_address + - mem_pub_info->offset), - (void *) ((uint64_t) mpi_global_data.one_sided_memory + - mem_pub_info->offset), mem_pub_info->size); - } - - free(mem_pub_info); - - triton_mutex_lock(&mpi_global_data.mutex); - mem_reg_info->publish_count--; - triton_mutex_unlock(&mpi_global_data.mutex); - - return TRITON_SUCCESS; -} - - -static int get_serialized_handle_size( - triton_serialized_handle_type_t handle_type, - void *handle) -{ - switch (handle_type) - { - case MEM_PUB_HANDLE_TYPE: - return (int) sizeof(mem_pub_t); - default: - triton_assert("unknown triton_serialized_handle_type_t"); - } - return -1; // this should never happen -} - - -static triton_ret_t get_serialized_handle( - triton_serialized_handle_type_t handle_type, - void *pointer_to_in_handle, - char *serialized_handle) -{ - mem_pub_t *mem_pub = NULL; - uint64_t *pu64; - int *pi; - switch (handle_type) - { - case MEM_PUB_HANDLE_TYPE: - mem_pub = *((mem_pub_t **) pointer_to_in_handle); - memcpy(serialized_handle, mem_pub, sizeof(mem_pub_t)); - memset(serialized_handle, 0, 2 * sizeof(void *)); // the 1st 2 - // fields are - // meaningless - // for remote - // procs - pu64 = (uint64_t *) (serialized_handle + 2 * sizeof(void *)); // offset - *pu64 = aehton64(*pu64); - pu64 = (uint64_t *) (serialized_handle + 2 * sizeof(void *) + sizeof(uint64_t)); // size - *pu64 = aehton64(*pu64); - pi = (int *) ((char *) pu64 + sizeof(uint64_t) + sizeof(int)); // client_rank - *pi = aehton32(*pi); - break; - default: - return TRITON_ERR_INVAL; - } - return TRITON_SUCCESS; -} - -static triton_ret_t create_handle_from_serialization( - triton_serialized_handle_type_t handle_type, - void *pointer_to_out_handle, - char *serialized_handle) -{ - mem_pub_t *mem_pub = NULL; - switch (handle_type) - { - case MEM_PUB_HANDLE_TYPE: - mem_pub = (mem_pub_t *) calloc(1, sizeof(mem_pub_t)); - if (!mem_pub) - return TRITON_ERR_NOMEM; - memcpy(mem_pub, serialized_handle, sizeof(mem_pub_t)); - mem_pub->offset = aentoh64(mem_pub->offset); - mem_pub->size = aentoh64(mem_pub->size); - mem_pub->client_rank = aentoh32(mem_pub->client_rank); - mem_pub->made_from_deserialization = 1; - *((triton_mem_pub_handle_t *) pointer_to_out_handle) = - (triton_mem_pub_handle_t) mem_pub; - break; - default: - return TRITON_ERR_INVAL; - } - return TRITON_SUCCESS; -} - -static triton_ret_t free_serialized_handle( - triton_serialized_handle_type_t handle_type, - void *pointer_to_in_handle) -{ - mem_pub_t *mem_pub = NULL; - switch (handle_type) - { - case MEM_PUB_HANDLE_TYPE: - mem_pub = *((mem_pub_t **) pointer_to_in_handle); - if (!mem_pub->made_from_deserialization) - return TRITON_ERR_INVAL; // it is forbidden to call this - // function on a handle - // that was not created through serialization - free(mem_pub); - *((triton_mem_pub_handle_t *) pointer_to_in_handle) = 0; - break; - default: - return TRITON_ERR_INVAL; - } - return TRITON_SUCCESS; -} - -static __blocking triton_ret_t get( - triton_msg_method_addr_t from, - triton_mem_pub_handle_t handle, - triton_iov_item_t * iovs, - char **dest_bufs, - int iov_items_count) -{ - int ret; - MPI_Datatype origin_hind_type; - MPI_Datatype target_hind_type; - int *lengths = NULL; - MPI_Aint *indices = NULL; - int i; - - if (iov_items_count == 0) - return TRITON_SUCCESS; - if (iov_items_count == 1) - { - ret = - MPI_Get(dest_bufs[0], iovs[0].size, MPI_CHAR, (int) from, - iovs[0].offset, iovs[0].size, MPI_CHAR, - mpi_global_data.wins[(int) from]); - RETURN_ON_MPI_FAILURE(ret, "MPI_Get failed"); - return TRITON_SUCCESS; - } - - lengths = malloc(iov_items_count * sizeof(int)); - if (!lengths) - return TRITON_ERR_NOMEM; - indices = malloc(iov_items_count * sizeof(MPI_Aint)); - if (!indices) - { - free(lengths); - return TRITON_ERR_NOMEM; - } - for (i = 0; i < iov_items_count; i++) - { - lengths[i] = (int) iovs[i].size; - indices[i] = (MPI_Aint) iovs[i].offset; - } - ret = - MPI_Type_create_hindexed(iov_items_count, lengths, indices, MPI_CHAR, - &target_hind_type); - if (ret != MPI_SUCCESS) - { - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Type_create_hindexed"); - } - ret = MPI_Type_commit(&target_hind_type); - if (ret != MPI_SUCCESS) - { - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Type_commit"); - } - - for (i = 0; i < iov_items_count; i++) - MPI_Get_address(dest_bufs[i], &indices[i]); - ret = - MPI_Type_create_hindexed(iov_items_count, lengths, indices, MPI_CHAR, - &origin_hind_type); - if (ret != MPI_SUCCESS) - { - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Type_create_hindexed"); - } - ret = MPI_Type_commit(&origin_hind_type); - if (ret != MPI_SUCCESS) - { - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Type_commit"); - } - - ret = MPI_Get(MPI_BOTTOM, 1, origin_hind_type, (int) from, 0, - 1, target_hind_type, mpi_global_data.wins[(int) from]); - - MPI_Type_free(&target_hind_type); - MPI_Type_free(&origin_hind_type); - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Get failed"); - return TRITON_SUCCESS; -} - -static __blocking triton_ret_t put( - triton_msg_method_addr_t to, - triton_mem_pub_handle_t handle, - triton_iov_item_t * iovs, - char **src_bufs, - int iov_items_count) -{ - int ret; - MPI_Datatype origin_hind_type; - MPI_Datatype target_hind_type; - int *lengths = NULL; - MPI_Aint *indices = NULL; - int i; - - if (iov_items_count == 0) - return TRITON_SUCCESS; - if (iov_items_count == 1) - { - ret = - MPI_Put(src_bufs[0], iovs[0].size, MPI_CHAR, (int) to, - iovs[0].offset, iovs[0].size, MPI_CHAR, - mpi_global_data.wins[(int) to]); - RETURN_ON_MPI_FAILURE(ret, "MPI_Put failed"); - return TRITON_SUCCESS; - } - - lengths = malloc(iov_items_count * sizeof(int)); - if (!lengths) - return TRITON_ERR_NOMEM; - indices = malloc(iov_items_count * sizeof(MPI_Aint)); - if (!indices) - { - free(lengths); - return TRITON_ERR_NOMEM; - } - for (i = 0; i < iov_items_count; i++) - { - lengths[i] = (int) iovs[i].size; - indices[i] = (MPI_Aint) iovs[i].offset; - } - ret = - MPI_Type_create_hindexed(iov_items_count, lengths, indices, MPI_CHAR, - &target_hind_type); - if (ret != MPI_SUCCESS) - { - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Type_create_hindexed"); - } - ret = MPI_Type_commit(&target_hind_type); - if (ret != MPI_SUCCESS) - { - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Type_commit"); - } - - for (i = 0; i < iov_items_count; i++) - MPI_Get_address(src_bufs[i], &indices[i]); - ret = - MPI_Type_create_hindexed(iov_items_count, lengths, indices, MPI_CHAR, - &origin_hind_type); - if (ret != MPI_SUCCESS) - { - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Type_create_hindexed"); - } - ret = MPI_Type_commit(&origin_hind_type); - if (ret != MPI_SUCCESS) - { - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Type_commit"); - } - - ret = MPI_Put(MPI_BOTTOM, 1, origin_hind_type, (int) to, 0, - 1, target_hind_type, mpi_global_data.wins[(int) to]); - - MPI_Type_free(&target_hind_type); - MPI_Type_free(&origin_hind_type); - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Get failed"); - return TRITON_SUCCESS; -} - -static __blocking triton_ret_t start_epoch( - triton_mem_pub_handle_t handle) -{ - mem_pub_t *mem_pub = (mem_pub_t *) handle; - int ret; - locked_rank_t *lr = NULL; - triton_mutex_lock(&mpi_global_data.mutex); - lr = find_locked_rank(mem_pub->client_rank); - if (!lr) - lr = add_locked_rank(mem_pub->client_rank); - lr->lock_count++; - triton_mutex_unlock(&mpi_global_data.mutex); - triton_mutex_lock(&lr->mutex); - ret = MPI_Win_lock(MPI_LOCK_SHARED, mem_pub->client_rank, 0, - mpi_global_data.wins[mem_pub->client_rank]); - RETURN_ON_MPI_FAILURE(ret, "MPI_Win_lock failed"); - return TRITON_SUCCESS; -} - - -static __blocking triton_ret_t end_epoch( - triton_mem_pub_handle_t handle) -{ - mem_pub_t *mem_pub = (mem_pub_t *) handle; - int ret; - locked_rank_t *lr = NULL; - - ret = - MPI_Win_unlock(mem_pub->client_rank, - mpi_global_data.wins[mem_pub->client_rank]); - RETURN_ON_MPI_FAILURE(ret, "MPI_Win_unlock failed"); - - triton_mutex_lock(&mpi_global_data.mutex); - lr = find_locked_rank(mem_pub->client_rank); - triton_assert(lr); // the lr MUST exist here! - lr->lock_count--; - triton_mutex_unlock(&lr->mutex); - if (lr->lock_count == 0) - remove_locked_rank(mem_pub->client_rank); - triton_mutex_unlock(&mpi_global_data.mutex); - - return TRITON_SUCCESS; -} - - -static __blocking triton_ret_t wait( - triton_mem_pub_handle_t mem_handle, - void *wait_info) -{ - // Noop. . . For this net-module, the orchestration of the - // completion guarantee is 100% managed in the upper layer - return TRITON_SUCCESS; -} - -static struct triton_msg_method triton_msg_mpi_method = { - .name = "mpi", - - .addr_self = mpi_addr_self, - .addr_equal = mpi_addr_equal, - .addr_to_string = mpi_addr_to_string, - .addr_lookup = mpi_addr_lookup, - .addr_free = mpi_addr_free, - .addr_copy = mpi_addr_copy, - .new_tag = mpi_new_tag, - - .group_add = mpi_group_add, - .send = mpi_send, - .recv = mpi_recv, - .recv_any = mpi_recv_any, - - .set_one_sided_threshold = set_one_sided_threshold, - .get_one_sided_threshold = get_one_sided_threshold, - .buffer_allocate = buffer_allocate, - .buffer_free = buffer_free, - .register_memory = register_memory, - .unregister_memory = unregister_memory, - .publish_memory = publish_memory, - .unpublish_memory = unpublish_memory, - .get_serialized_handle_size = get_serialized_handle_size, - .create_handle_from_serialization = create_handle_from_serialization, - .get_serialized_handle = get_serialized_handle, - .free_serialized_handle = free_serialized_handle, - .get = get, - .put = put, - .start_epoch = start_epoch, - .end_epoch = end_epoch, - .wait = wait -}; - -triton_ret_t triton_msg_mpi_init( - void) -{ - int ret; - int i; - int env_value_internal_buf_size; - char *env_value_str; - triton_ret_t tret = TRITON_SUCCESS; - int mpi_thread_level; - - triton_mutex_lock(&module_lock); - - if (!module_refcount) - { - MPI_Initialized(&mpi_already_up); - if(!mpi_already_up) - { - ret = MPI_Init_thread(NULL, NULL, MPI_THREAD_MULTIPLE, &mpi_thread_level); - assert(ret == MPI_SUCCESS); - mpi_did_init = 1; - } - - tret = triton_msg_init(); - if (tret != TRITON_SUCCESS) - { - if(mpi_did_init) - MPI_Finalize(); - triton_mutex_unlock(&module_lock); - return (tret); - } - - ret = aesop_sem_module_init(); - if(ret != 0) - { - if(mpi_did_init) - MPI_Finalize(); - triton_msg_finalize(); - triton_mutex_unlock(&module_lock); - return (tret); - } - - ret = MPI_Comm_rank(MPI_COMM_WORLD, &mpi_global_data.rank); - if (ret != MPI_SUCCESS) - { - if(mpi_did_init) - MPI_Finalize(); - aesop_sem_module_finalize(); - triton_msg_finalize(); - triton_mutex_unlock(&module_lock); - return (triton_error_from_mpi(ret)); - } - - mpi_rank = (triton_msg_method_addr_t) mpi_global_data.rank; - ret = MPI_Comm_size(MPI_COMM_WORLD, &mpi_global_data.size); - if (ret != MPI_SUCCESS) - { - if(mpi_did_init) - MPI_Finalize(); - aesop_sem_module_finalize(); - triton_msg_finalize(); - triton_mutex_unlock(&module_lock); - return (triton_error_from_mpi(ret)); - } - - env_value_str = getenv(ENV_INTERNAL_ONE_SIDED_BUFFER_SIZE); - if (env_value_str) - { - env_value_internal_buf_size = atoi(env_value_str); - if (env_value_internal_buf_size) - mpi_internal_one_sided_buffer_size = - (uint32_t) env_value_internal_buf_size; - } - - - // Create the Win for one-sided; as well as the machinery for mem - // registration - triton_mutex_init(&mpi_global_data.mutex, NULL); - mpi_global_data.one_sided_memory = - malloc(mpi_internal_one_sided_buffer_size); - ABORT_IF(mpi_global_data.one_sided_memory == NULL, - "Memory allocation failed"); - - mpi_global_data.remaining_mem_size = mpi_internal_one_sided_buffer_size; - mpi_global_data.mem_range_head = - (mem_range_t *) calloc(1, sizeof(mem_range_t)); - ABORT_IF(mpi_global_data.mem_range_head == NULL, - "Memory allocation failed"); - - mpi_global_data.mem_range_head->offset = 0; - mpi_global_data.mem_range_head->size = - mpi_internal_one_sided_buffer_size; - mpi_global_data.mem_range_head->mem_reg_info = NULL; - - mpi_global_data.wins = malloc(sizeof(MPI_Win) * mpi_global_data.size); - ABORT_IF(mpi_global_data.wins == NULL, "Memory allocation failed"); - - for (i = 0; i < mpi_global_data.size; i++) - { - ret = - MPI_Win_create(mpi_global_data.one_sided_memory, - mpi_internal_one_sided_buffer_size, sizeof(char), - MPI_INFO_NULL, MPI_COMM_WORLD, - &mpi_global_data.wins[i]); - ABORT_ON_MPI_FAILURE(ret, "MPI_Win_create failed"); - } - - assert(group_table == NULL); - group_table = - triton_hash_init(mpi_group_compare, triton_hash_32bit_hash, 1024); - if (!group_table) - { - triton_err(triton_log_default, - "%s:%d: Failed to initialize group hashtable", __FILE__, - __LINE__); - assert(group_table); - } - - thread_stop_flag = 0; - ret = pthread_create(&thread_id, NULL, thread_fn, NULL); - /* TODO: error handling */ - assert(ret == 0); - - tret = triton_msg_method_register(&triton_msg_mpi_method); - if (tret != TRITON_SUCCESS) - { - /* TODO: more cleanup/error handling */ - assert(0); - triton_mutex_unlock(&module_lock); - return (tret); - } - } - module_refcount++; - triton_mutex_unlock(&module_lock); - - return TRITON_SUCCESS; -} - -static void internal_finalize( - void) -{ - int i; - int mpi_finalized; - - triton_msg_method_unregister("mpi"); - - MPI_Finalized(&mpi_finalized); - - /* NOTE: this is a workaround; we can't cleanly make any further MPI - * calls if an aesop callback has already executed a direct call to - * MPI_Finalize() before we reach this point. - */ - if(!mpi_finalized) - { - if (group_table != NULL) - { - triton_hash_destroy_and_finalize(group_table, - struct triton_mpi_group, - link, - group_entry_free); - group_table = NULL; - } - for (i = 0; i < mpi_global_data.size; i++) - MPI_Win_free(&mpi_global_data.wins[i]); - free(mpi_global_data.one_sided_memory); - triton_assert(mpi_global_data.mem_range_head->next == NULL); - free(mpi_global_data.mem_range_head); - triton_mutex_destroy(&mpi_global_data.mutex); - } - - triton_msg_finalize(); - aesop_sem_module_finalize(); - if(mpi_did_init) - { - MPI_Finalize(); - } - - return; -} - - -void triton_msg_mpi_finalize( - void) -{ - triton_mutex_lock(&module_lock); - module_refcount--; - - if (!module_refcount) - { - triton_mutex_lock(&thread_mutex); - thread_stop_flag = 1; - triton_mutex_unlock(&thread_mutex); - wake_mpi_thread(); - } - triton_mutex_unlock(&module_lock); -} - -static MPI_Request *req_array = NULL; -static int *index_array = NULL; -static MPI_Status *status_array = NULL; -static struct nb_request **nb_req_array = NULL; -static int array_max = 0; -static int array_size = 0; - -static void realloc_if_needed(int additional) -{ - if(array_max == 0 || (array_size + additional > array_max)) - { - array_max += 64; - req_array = realloc(req_array, array_max*sizeof(*req_array)); - assert(req_array); - index_array = realloc(index_array, array_max*sizeof(*index_array)); - assert(index_array); - status_array = realloc(status_array, array_max*sizeof(*status_array)); - assert(status_array); - nb_req_array = realloc(nb_req_array, array_max*sizeof(*nb_req_array)); - assert(nb_req_array); - } -} - -static void wake_mpi_thread(void) -{ - triton_mutex_lock(&thread_mutex); - if(thread_wake_flag) - { - triton_mutex_unlock(&thread_mutex); - return; - } - thread_wake_flag = 1; - triton_mutex_unlock(&thread_mutex); - - MPI_Send(NULL, 0, MPI_BYTE, 0, 0, MPI_COMM_SELF); -} - -static void* thread_fn(void* arg) -{ - triton_list_link_t *tmp_link; - struct nb_request *nb_req; - int ret; - int outcount; - int i; - int cancelled; - int mpi_finalized = 0; - - /* initialize arrays */ - realloc_if_needed(1); - - /* first iteration; post recv from ourself to wake up thread - * when needed. - */ - MPI_Irecv(NULL, 0, MPI_BYTE, 0, 0, MPI_COMM_SELF, &req_array[0]); - nb_req_array[0] = NULL; - array_size = 1; - - triton_mutex_lock(&thread_mutex); - while(!thread_stop_flag) - { - - realloc_if_needed(triton_list_count(&posted_queue)); - - tmp_link = triton_queue_dequeue(&posted_queue); - while(tmp_link) - { - nb_req = triton_list_get_entry(tmp_link, struct nb_request, link); - assert(nb_req); - - nb_req_array[array_size] = nb_req; - req_array[array_size] = nb_req->request; - array_size++; - triton_list_add_back(tmp_link, &inflight_queue); - tmp_link = triton_queue_dequeue(&posted_queue); - } - - triton_mutex_unlock(&thread_mutex); - - ret = MPI_Waitsome(array_size, req_array, &outcount, index_array, - status_array); - assert(ret == MPI_SUCCESS || ret == MPI_ERR_IN_STATUS); - - /* loop backwarks through array so that we can safely move later - * operations up to new positions in the array - */ - for(i = outcount-1; i >= 0; i--) - { - /* NOTE: this is a workaround to a peculiar aesop callback race - * condition. It is possible for the aesop_sem_down() call - * below to invoke an aesop callback that runs MPI_Finalize() in - * the caller's code context. In that case we can no longer - * safely make any MPI_* calls here. - */ - MPI_Finalized(&mpi_finalized); - - if(index_array[i] == 0 && !thread_stop_flag && !mpi_finalized) - { - /* special message from self to wait up MPI_Waitsome(). - * Ignore the status and post a new recv to take its place - */ - MPI_Irecv(NULL, 0, MPI_BYTE, 0, 0, MPI_COMM_SELF, &req_array[0]); - triton_mutex_lock(&thread_mutex); - thread_wake_flag = 0; - triton_mutex_unlock(&thread_mutex); - } - else if(index_array[i] != 0) - { - nb_req = nb_req_array[index_array[i]]; - /* figure out return code */ - if(thread_stop_flag || mpi_finalized) - cancelled = 1; - else - MPI_Test_cancelled(&status_array[i], &cancelled); - if(cancelled) - nb_req->tret = TRITON_ERR_CANCELED; - else if(ret == MPI_ERR_IN_STATUS) - nb_req->tret = triton_error_from_mpi(status_array[i].MPI_ERROR); - else - nb_req->tret = TRITON_SUCCESS; - - /* if this was a successful receive, fill in bytes, from, - * and tag */ - if(nb_req->tret == TRITON_SUCCESS && nb_req->bytes_received) - { - MPI_Get_elements(&status_array[i], nb_req->dtype, &nb_req->bytes_received); - nb_req->from = status_array[i].MPI_SOURCE; - nb_req->tag = status_array[i].MPI_TAG; - } - - /* TODO: is this locking right? walk throug this again... */ - triton_mutex_lock(&thread_mutex); - /* put in completed queue */ - triton_list_del(&nb_req->link); - triton_list_add_back(&nb_req->link, &completed_queue); - triton_mutex_unlock(&thread_mutex); - - /* adjust the arrays to reflect that we completed an operation */ - if(index_array[i] == array_size-1) - { - /* this was the last request in the array, clear it out */ - nb_req_array[index_array[i]] = NULL; - req_array[index_array[i]] = MPI_REQUEST_NULL; - } - else - { - /* move the last request down to this position */ - nb_req_array[index_array[i]] = - nb_req_array[array_size-1]; - req_array[index_array[i]] = - req_array[array_size-1]; - } - array_size--; - - if(!thread_stop_flag && !mpi_finalized) - MPI_Type_free(&nb_req->dtype); - - aesop_sem_up(&nb_req->sem); - } - } - triton_mutex_lock(&thread_mutex); - } - triton_mutex_unlock(&thread_mutex); - - internal_finalize(); - - return(NULL); -} - -#define MPI_UPPER_FLAG 1 -#define rank2node(rank, node) ((node).l = (rank), (node).u = MPI_UPPER_FLAG) -static triton_ret_t triton_error_from_mpi(int r) -{ - int size; - char mpi_error_msg[MPI_MAX_ERROR_STRING]; - int rank, class; - - if(r == MPI_SUCCESS) - { - return TRITON_SUCCESS; - } - - MPI_Error_class(r, &class); - if(class == MPI_ERR_UNKNOWN) - { - /* maybe was canceled */ - return triton_error_from_errno(r); - } - - memset(mpi_error_msg, 0, MPI_MAX_ERROR_STRING); - - MPI_Error_string(r, mpi_error_msg, &size); - MPI_Comm_rank(MPI_COMM_WORLD, &rank); - return triton_error_wrap(TRITON_ERR_MPI, triton_node_null, "%s", mpi_error_msg); -} - - -/* - * Local variables: - * c-indent-level: 4 - * c-basic-offset: 4 - * End: - * - * vim: ft=c ts=8 sts=4 sw=4 expandtab - */ diff --git a/code/src/net/mpi/mpi-method.h b/code/src/net/mpi/mpi-method.h deleted file mode 100644 index 3240bec..0000000 --- a/code/src/net/mpi/mpi-method.h +++ /dev/null @@ -1,11 +0,0 @@ -#ifndef __MPI_METHOD_H__ -#define __MPI_METHOD_H__ - -#include "src/common/triton-error.h" - -triton_ret_t triton_msg_mpi_init(void); -void triton_msg_mpi_finalize(void); - -triton_ret_t triton_mpi_init_client(void); -void triton_mpi_finalize_client(void); -#endif /* __MPI_METHOD_H__ */ diff --git a/code/src/net/mpi/one_sided_interface_mpiimpl_2/README b/code/src/net/mpi/one_sided_interface_mpiimpl_2/README deleted file mode 100644 index d657f3c..0000000 --- a/code/src/net/mpi/one_sided_interface_mpiimpl_2/README +++ /dev/null @@ -1,6 +0,0 @@ -This is a standalone test. It tests the one-sided interface implementation of the MPI net-module - -./build to build -Then run as an MPI program - -Provide a command-line argument (anything) to the program to display the help diff --git a/code/src/net/mpi/one_sided_interface_mpiimpl_2/build b/code/src/net/mpi/one_sided_interface_mpiimpl_2/build deleted file mode 100755 index 7d15808..0000000 --- a/code/src/net/mpi/one_sided_interface_mpiimpl_2/build +++ /dev/null @@ -1 +0,0 @@ -mpicc -ggdb -o test test.c mock.c mpi-message-method.c diff --git a/code/src/net/mpi/one_sided_interface_mpiimpl_2/mock.c b/code/src/net/mpi/one_sided_interface_mpiimpl_2/mock.c deleted file mode 100644 index aa15ec7..0000000 --- a/code/src/net/mpi/one_sided_interface_mpiimpl_2/mock.c +++ /dev/null @@ -1,16 +0,0 @@ -#include "mock.h" -#include <stdio.h> - -void __jz_wait_for_debugger(const char* file, const char* function, int line, int rank, int once) -{ - static int count = 0; - - if(count && once) - return; - count++; - printf("process ( pid = %d | rank = %d ) is waiting in %s at %s:%d for debugger\n", - getpid(), rank, function, file, line); - fflush(stdout); - for(;;); -} - diff --git a/code/src/net/mpi/one_sided_interface_mpiimpl_2/mock.h b/code/src/net/mpi/one_sided_interface_mpiimpl_2/mock.h deleted file mode 100644 index 9978b3f..0000000 --- a/code/src/net/mpi/one_sided_interface_mpiimpl_2/mock.h +++ /dev/null @@ -1,134 +0,0 @@ - -#ifndef __JZ_MPI_TEST_MOCK__ -#define __JZ_MPI_TEST_MOCK__ - - -#define JZ_DEBUG - -#define __JZ -#include <unistd.h> -#include "assert.h" -#ifndef uint64_t -typedef unsigned long long uint64_t; -#endif -#ifndef uint32_t -typedef unsigned int uint32_t; -#endif - - -#define MIN(a,b) ((a)<=(b)?(a):(b)) -#define MAX(a,b) ((a)>=(b)?(a):(b)) - -#define __JZ_TEMP_CODE - -#ifdef __JZ_TEMP_CODE -#define JZ_TEMP_INSTRUCTION(a) a -#define JZ_START_TEMP_CODE_BLOCK do{if(1) -#define JZ_END_TEMP_CODE_BLOCK }while(0); //the semi-colon is not a mistake -#else -#define JZ_TEMP_INSTRUCTION(a) -#define JZ_START_TEMP_CODE_BLOCK do{if(0) -#define JZ_END_TEMP_CODE_BLOCK }while(0); -#endif - -#define JZ_PRINT(...) do{printf(__VA_ARGS__); printf("\n"); fflush(stdout);}while(0) -#define JZ_PRINT_IF(cond,...) do{if(cond){printf(__VA_ARGS__); printf("\n"); fflush(stdout);}}while(0) - -#define PRINT(...) do{ printf(__VA_ARGS__); fflush(stdout);}while(0) - -#define RED_TEXT "\x1b[31m" -#define GREEN_TEXT "\x1b[32m" -#define NEUTRAL_TEXT "\x1b[0m" - -#define UNIT_TEST(cond) \ -do{PRINT("[pid %d][rank %d][%s:%s:%d] %s\n", getpid(), (int)mpi_rank, __FILE__, \ - __FUNCTION__, __LINE__, (cond)? GREEN_TEXT "PASSED" NEUTRAL_TEXT \ - :RED_TEXT "FAILED!!!!" NEUTRAL_TEXT);} while(0) - - -#define JZ_DUMMY //[JZ] Readability and TODO hint. just to decorate thinks that are dummy or temporary - - - -#define JZ_REPORT_ERR(...) \ -do{ \ - fprintf(stderr, "[in %s at %s:%d]", __FUNCTION__, __FILE__, __LINE__); \ - fprintf(stderr, __VA_ARGS__); fprintf(stderr, "\n"); fflush(stderr); \ -}while(0) - -#define TRITON_SUCCESS 0 -#define TRITON_ERR_NOMEM -1 -#define TRITON_ERR_INVAL -2 -#define TRITON_ERR_MEM_IN_USE -3 -#define TRITON_ERR_MEM_OUT_OF_RANGE -4 -#define TRITON_ERR_UNKNOWN -5 -#define TRITON_ERR_MAX -6 - - -#define TRITON_MUTEX_INITIALIZER 0 - -#define triton_err(a,...) fprintf(stderr, __VA_ARGS__) -#define triton_error_from_mpi(ret) 1 -#define triton_hash_init(...) (void*)0xdeadbeef -#define triton_hash_32bit_hash(...) (void*)0xdeadbeef -#define triton_mutex_destroy(...) -#define triton_assert(cond) assert((cond)) -typedef int triton_ret_t; - - -#define triton_mutex_t int -#define triton_mutex_lock(a) -#define triton_mutex_unlock(a) -#define triton_mutex_init(...) - -#define triton_init_register(...) -#define triton_hash_destroy_and_finalize(...) - -#define __blocking -#define pwait -#define pbranch -#define pprivate -#define aesop_cancel_branches() -#define aesop_cancel_branches_wait() - -struct triton_hash_table -{ -}; - -typedef struct triton_hash_link -{ -}triton_hash_link; - -typedef const char* triton_string_t; -typedef uint32_t triton_msg_tag_t; - -triton_ret_t triton_mpi_init(void); -void triton_mpi_finalize(void); - -triton_ret_t triton_mpi_init_client(); -void triton_mpi_finalize_client(); -triton_ret_t triton_msg_mpi_init(void); -void triton_msg_mpi_finalize(void); - - - -#ifdef JZ_DEBUG - void __jz_wait_for_debugger(const char* file, const char* function, int line, int rank, int once); - #define JZ_WAIT_FOR_DEBUGGER() __jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 0) - #define JZ_WAIT_FOR_DEBUGGER_IF(cond) \ - do{if((cond))__jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 0);}while(0) - #define JZ_WAIT_ONCE_FOR_DEBUGGER() __jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 1) - #define JZ_WAIT_ONCE_FOR_DEBUGGER_IF(cond) \ - do{if((cond))__jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 1);}while(0) -#else - #define JZ_WAIT_FOR_DEBUGGER() - #define JZ_WAIT_FOR_DEBUGGER_IF() - #define JZ_WAIT_ONCE_FOR_DEBUGGER() - #define JZ_WAIT_ONCE_FOR_DEBUGGER_IF() -#endif - - struct triton_msg_method; - struct triton_msg_method* get_msg_method(); - - void test_mem_reg_pub(); -#endif //__JZ_MPI_TEST_MOCK__ diff --git a/code/src/net/mpi/one_sided_interface_mpiimpl_2/mpi-message-method.c b/code/src/net/mpi/one_sided_interface_mpiimpl_2/mpi-message-method.c deleted file mode 100644 index d390f69..0000000 --- a/code/src/net/mpi/one_sided_interface_mpiimpl_2/mpi-message-method.c +++ /dev/null @@ -1,1433 +0,0 @@ -#include "mock.h" -#include "triton-message-method.h" - - - -#ifndef __JZ - -#include "src/net/mpi/mpi-method.h" -#include <triton-hash.h> - -#include "src/net/mpi/mpi.hae" - -#include "src/net/triton-addr.h" -#include "src/net/mpi/errors.h" -#include "src/net/triton-message.hae" -#include "src/net/triton-message-method.hae" -#include "src/net/mpi/mpi-method.h" -#endif //__JZ - -#include <errno.h> -#include <mpi.h> -#include <stdio.h> -#include <assert.h> -#include <stdlib.h> -#include <string.h> -#include <stdarg.h> -#include <pthread.h> -#include <mpi.h> -#include <unistd.h> - -static triton_msg_method_addr_t mpi_rank = -1; - -static struct triton_hash_table *group_table = NULL; - -static __blocking triton_msg_method_addr_t mpi_addr_self(void) -{ - return mpi_rank; -} - -static int mpi_addr_equal(triton_msg_method_addr_t a1, triton_msg_method_addr_t a2) -{ - return a1 == a2; -} - -static triton_ret_t mpi_addr_to_string(triton_msg_method_addr_t addr, triton_string_t *addrstr) -{ -#if 0 //JZ - triton_string_init(addrstr, "%u", (uint32_t)addr); -#endif //JZ - return TRITON_SUCCESS; -} - -static triton_ret_t mpi_addr_free(triton_msg_method_addr_t addr) -{ - //Nothing to do - return TRITON_SUCCESS; -} - -static triton_ret_t mpi_addr_copy(const triton_msg_method_addr_t orig_addr, triton_msg_method_addr_t *copy) -{ - *copy = orig_addr; - return TRITON_SUCCESS; -} - -static triton_ret_t mpi_addr_lookup(const char *name, triton_msg_method_addr_t *addr) -{ - *addr = atoi(name); - return TRITON_SUCCESS; -} - -struct triton_mpi_group -{ - char *name; - triton_msg_method_group_t group; - MPI_Comm comm; - int tag_counter; - triton_mutex_t mutex; - struct triton_hash_link link; -}; - -static triton_ret_t mpi_group_add(const char *service_name, triton_msg_method_group_t *group) -{ -#if 0 //JZ - int ret; - uint32_t h1 = 0, h2 = 0; - struct triton_mpi_group *newgroup; - - assert(service_name); - assert(group); - - newgroup = malloc(sizeof(*newgroup)); - if(newgroup == NULL) return TRITON_ERR_NOMEM; - - newgroup->name = strdup(service_name); - - bj_hashlittle2(service_name, strlen(service_name), &h1, &h2); - newgroup->group = h1; - newgroup->tag_counter = TRITON_MPI_TAG_MIN; - triton_mutex_init(&newgroup->mutex, NULL); - *group = newgroup->group; - - ret = MPI_Comm_dup(MPI_COMM_WORLD, &newgroup->comm); - if(ret != MPI_SUCCESS) - { - return triton_error_from_mpi(ret); - } - - triton_list_link_clear(&newgroup->link); - triton_hash_add(group_table, &newgroup->group, &newgroup->link); -#endif //JZ - return TRITON_SUCCESS; -} - -static triton_msg_tag_t mpi_new_tag(triton_msg_method_group_t group) -{ -#if 0 //JZ - struct triton_hash_link *groupl; - struct triton_mpi_group *group_entry; - triton_msg_tag_t tag; - - /* lookup communicator */ - groupl = triton_hash_search(group_table, &group); - group_entry = triton_hash_get_entry(groupl, struct triton_mpi_group, link); - triton_mutex_lock(&group_entry->mutex); - ++(group_entry->tag_counter); - if(group_entry->tag_counter >= MPI_TAG_UB) - { - group_entry->tag_counter = TRITON_MPI_TAG_MIN; - } - tag = group_entry->tag_counter; - triton_mutex_unlock(&group_entry->mutex); - return tag; -#endif //JZ - return 0; -} - - -static __blocking triton_ret_t mpi_send( - triton_msg_method_addr_t to, - triton_msg_method_group_t group, - triton_msg_tag_t tag, - int count, - char **buffers, - uint32_t *sizes) -{ -#if 0 //JZ - int ret, i; - MPI_Datatype hind_type; - MPI_Aint *offsets; - struct triton_hash_link *groupl; - struct triton_mpi_group *group_entry; - - offsets = malloc(sizeof(*offsets) * count); - if(offsets == NULL) - { - return TRITON_ERR_NOMEM; - } - - for(i = 0; i < count; ++i) - { - MPI_Get_address(buffers[i], &offsets[i]); - } - - ret = MPI_Type_create_hindexed(count, (int *)sizes, offsets, MPI_CHAR, &hind_type); - if(ret != MPI_SUCCESS) - { - free(offsets); - return triton_error_from_mpi(ret); - } - - ret = MPI_Type_commit(&hind_type); - if(ret != MPI_SUCCESS) - { - free(offsets); - MPI_Type_free(&hind_type); - return triton_error_from_mpi(ret); - } - - /* lookup communicator */ - groupl = triton_hash_search(group_table, &group); - group_entry = triton_hash_get_entry(groupl, struct triton_mpi_group, link); - - ret = triton_mpi_send(MPI_BOTTOM, 1, hind_type, (int)to, tag, group_entry->comm); - if(ret != MPI_SUCCESS) - { - free(offsets); - MPI_Type_free(&hind_type); - return triton_error_from_mpi(ret); - } - - free(offsets); - MPI_Type_free(&hind_type); - return TRITON_SUCCESS; -#endif //JZ - return TRITON_SUCCESS; -} - -static __blocking triton_ret_t mpi_recv( - triton_msg_method_addr_t from, - triton_msg_method_group_t group, - triton_msg_tag_t tag, - int count, - char **buffers, - uint32_t *sizes, - uint32_t *bytes_received) -{ -#if 0 - int ret, i; - MPI_Datatype hind_type; - MPI_Aint *offsets; - struct triton_hash_link *groupl; - struct triton_mpi_group *group_entry; - MPI_Status status; - - offsets = malloc(sizeof(*offsets) * count); - if(offsets == NULL) - { - return TRITON_ERR_NOMEM; - } - - for(i = 0; i < count; ++i) - { - MPI_Get_address(buffers[i], &offsets[i]); - } - - ret = MPI_Type_create_hindexed(count, (int *)sizes, offsets, MPI_CHAR, &hind_type); - if(ret != MPI_SUCCESS) - { - free(offsets); - return triton_error_from_mpi(ret); - } - - ret = MPI_Type_commit(&hind_type); - if(ret != MPI_SUCCESS) - { - free(offsets); - MPI_Type_free(&hind_type); - return triton_error_from_mpi(ret); - } - - /* lookup communicator */ - groupl = triton_hash_search(group_table, &group); - group_entry = triton_hash_get_entry(groupl, struct triton_mpi_group, link); - - ret = triton_mpi_recv(MPI_BOTTOM, 1, hind_type, (int)from, tag, group_entry->comm, &status); - if(ret != MPI_SUCCESS) - { - int canceled; - free(offsets); - MPI_Type_free(&hind_type); - MPI_Test_cancelled(&status, &canceled); - if(canceled) - { - return TRITON_ERR_CANCELED; - } - - return triton_error_from_mpi(ret); - } - - ret = MPI_Get_elements(&status, hind_type, (int *)bytes_received); - if(ret != MPI_SUCCESS) - { - free(offsets); - MPI_Type_free(&hind_type); - return triton_error_from_mpi(ret); - } - - free(offsets); - MPI_Type_free(&hind_type); - return TRITON_SUCCESS; -#endif //JZ - return TRITON_SUCCESS; -} - -static __blocking triton_ret_t mpi_recv_any( - triton_msg_method_group_t group, - triton_msg_method_addr_t *from, - triton_msg_tag_t *tag, - int count, - char **buffers, - uint32_t *sizes, - uint32_t *bytes_received) -{ -#if 0 //JZ - int ret, i; - MPI_Datatype hind_type; - MPI_Aint *offsets; - struct triton_hash_link *groupl; - struct triton_mpi_group *group_entry; - MPI_Status status; - - offsets = malloc(sizeof(*offsets) * count); - if(offsets == NULL) - { - return TRITON_ERR_NOMEM; - } - - for(i = 0; i < count; ++i) - { - MPI_Get_address(buffers[i], &offsets[i]); - } - - ret = MPI_Type_create_hindexed(count, (int *)sizes, offsets, MPI_CHAR, &hind_type); - if(ret != MPI_SUCCESS) - { - free(offsets); - return triton_error_from_mpi(ret); - } - - ret = MPI_Type_commit(&hind_type); - if(ret != MPI_SUCCESS) - { - free(offsets); - MPI_Type_free(&hind_type); - return triton_error_from_mpi(ret); - } - - /* lookup communicator */ - groupl = triton_hash_search(group_table, &group); - group_entry = triton_hash_get_entry(groupl, struct triton_mpi_group, link); - - ret = triton_mpi_recv(MPI_BOTTOM, 1, hind_type, MPI_ANY_SOURCE, MPI_ANY_TAG, group_entry->comm, &status); - if(ret != MPI_SUCCESS) - { - int canceled; - free(offsets); - MPI_Type_free(&hind_type); - MPI_Test_cancelled(&status, &canceled); - if(canceled) - { - return TRITON_ERR_CANCELED; - } - - return triton_error_from_mpi(ret); - } - - ret = MPI_Get_elements(&status, hind_type, (int *)bytes_received); - if(ret != MPI_SUCCESS) - { - free(offsets); - MPI_Type_free(&hind_type); - return triton_error_from_mpi(ret); - } - - *from = status.MPI_SOURCE; - - *tag = status.MPI_TAG; - - free(offsets); - MPI_Type_free(&hind_type); - return TRITON_SUCCESS; -#endif //JZ - return TRITON_SUCCESS; -} - -static int mpi_group_compare(const void *key, struct triton_hash_link *link) -{ -#if 0 - const triton_msg_method_group_t *group1 = (const triton_msg_method_group_t *)key; - struct triton_mpi_group *mpi_group = triton_hash_get_entry(link, struct triton_mpi_group, link); - assert(key); - assert(link); - - return (mpi_group->group == *group1); -#endif - return 1; -} - -static int mpi_enabled = 0; - -static triton_ret_t mpi_enable(void) -{ - mpi_enabled = 1; - return TRITON_SUCCESS; -} - -static void group_entry_free(void *x) -{ -#if 0 //JZ - struct triton_mpi_group *g; - - g = (struct triton_mpi_group *)x; - - triton_mutex_destroy(&g->mutex); - MPI_Comm_free(&g->comm); - free(g->name); - free(g); -#endif //JZ -} - -static void mpi_disable(void) -{ - mpi_enabled = 0; -} - -#define RETURN_ON_MPI_FAILURE(mpi_ret_code,err_msg_format,...) \ -do \ -{ \ - if((mpi_ret_code) != MPI_SUCCESS) \ - { \ - triton_err(triton_log_default, "%s:%d: [MPI Proc rank %d]" err_msg_format, __FILE__, \ - __LINE__, (int)mpi_rank, ##__VA_ARGS__); \ - return triton_error_from_mpi((ret)); \ - } \ -}while(0) - - - -/* -----------------------------------------------------------------------------------------------*/ -#define ABORT_ON_FAILURE(ret,err_msg_format,...) \ -do \ -{ \ - if((ret) != MPI_SUCCESS) \ - { \ - triton_err(triton_log_default, "%s:%d: [MPI Proc rank %d]" err_msg_format, \ - __FILE__, __LINE__, (int)mpi_rank, ##__VA_ARGS__); \ - MPI_Abort(MPI_COMM_WORLD, ret); \ - } \ -}while(0) - -#define KB 1024 -#define MB 1048576 -#define TRITON_MPI_TAG_MIN 100 -#define TRITON_MPI_MAX_USABLE_REGISTERED_MEM_SIZE (512*MB) -#define TRITON_MPI_REGISTERED_RMA_MEM_SIZE TRITON_MPI_MAX_USABLE_REGISTERED_MEM_SIZE -#define TRITON_MPI_REGISTERED_CONTROL_MESSAGE_SIZE sizeof(uint64_t) - -#define ZERO_SIZE_MEM_HANDLE 0x80000000CAFEBABE - -#define DEFAULT_MPI_ONE_SIDED_SIZE_THRESHOLD 0 - -#define MAX_INTERNAL_CONTROL_MESSAGE_SIZE KB - -static uint64_t mpi_one_sided_size_threshold = DEFAULT_MPI_ONE_SIDED_SIZE_THRESHOLD; -triton_ret_t set_one_sided_threshold(size_t size) -{ - triton_assert(size >= 0); - mpi_one_sided_size_threshold = size; - return TRITON_SUCCESS; -} - - -triton_ret_t get_one_sided_threshold(size_t *size) -{ - *size = (size_t)mpi_one_sided_size_threshold; - return TRITON_SUCCESS; -} - - -typedef struct mem_reg_t -{ - void* base_address; //address of the portion taken from the buffer exposed for one-sided - //this should be normally size+sizeof(uint64_t) long - void* user_address; /*address provided by the user. This pointer is the same as base_address if - the handle has been created through buffer_allocate - We use the following convention: - 1- base_address == user_address => reg_handle is associated to a - buffer_allocated memory and that memory has been registered as well - 2- user_address == NULL => reg_handle is associated to a buffer_allocated - and that memory has not been registered or is already unregistered - 3- base_address != user_address && user_address != NULL => - reg_handle is associated with a non-buffer_allocated memory - that is currently registered - */ - uint64_t size; - int publish_count; -}mem_reg_t; - -typedef struct mem_range_t -{ - struct mem_range_t* next; - mem_reg_t* mem_reg_info; - uint64_t offset; - uint64_t size; - int made_from_deserialization; //1 if made from deserialization - int client_rank; //the rank fo the peer that initially published the memory -}mem_range_t; - -typedef mem_range_t mem_pub_t; - -struct -{ - int rank; - MPI_Win win; - - void* one_sided_memory; //It is the memory registered for one-sided - uint64_t remaining_mem_size; // Exposed by clients to get server operation completion count - triton_mutex_t mutex; - mem_range_t* mem_range_head; // Head of memory slices. -}global_data = {-1, MPI_WIN_NULL, NULL, 0, TRITON_MUTEX_INITIALIZER, NULL}; - -/*Find the mem_range_t* associated with a certain address -*/ -static mem_range_t* find_associated_mem_range(void* address) -{ - mem_range_t* cur_mem_range_handle = NULL; - for(cur_mem_range_handle = global_data.mem_range_head; - cur_mem_range_handle; cur_mem_range_handle = cur_mem_range_handle->next) - { - if(cur_mem_range_handle->mem_reg_info && - cur_mem_range_handle->mem_reg_info->base_address == address) - return cur_mem_range_handle; - } - return NULL; -} - -static triton_ret_t register_memory_internal(void* base, - uint64_t size, - mem_reg_t** handle - ) -{ - mem_range_t* cur_mem_range_handle = NULL; - mem_range_t* prev_mem_range_handle = NULL; - mem_range_t* found_mem_range = NULL; - mem_reg_t* mem_reg_info = NULL; - *handle = NULL; - - /*Base might have been allocated with buffer_allocate in which case we only need - to return its associated handle*/ - mem_range_t *range = find_associated_mem_range(base); - if(range) - { - range->mem_reg_info->user_address = range->mem_reg_info->base_address; - *handle = range->mem_reg_info; - return TRITON_SUCCESS; - } - - //Check for space availability - if(size > global_data.remaining_mem_size) - { - return TRITON_ERR_NOMEM; - } - - triton_mutex_lock(&global_data.mutex); - - //check if any contiguous memory slice exists that is at least size + control region - cur_mem_range_handle = global_data.mem_range_head; - while(cur_mem_range_handle) - { - if(cur_mem_range_handle->mem_reg_info == NULL) //the range is not currently registered - { - if(cur_mem_range_handle->size > size) - { - //slice some memory for the registration - mem_range_t* new_mem_range_handle = (mem_range_t*)malloc(sizeof(mem_range_t)); - if(new_mem_range_handle == NULL) - { - triton_mutex_unlock(&global_data.mutex); - return TRITON_ERR_NOMEM; - } - mem_reg_info = (mem_reg_t*)malloc(sizeof(mem_reg_t)); - if(mem_reg_info == NULL) - { - triton_mutex_unlock(&global_data.mutex); - return TRITON_ERR_NOMEM; - } - mem_reg_info->publish_count = 0; - - new_mem_range_handle->offset = cur_mem_range_handle->offset; - new_mem_range_handle->size = size; - - cur_mem_range_handle->offset += size; - cur_mem_range_handle->size -= size; - - if(prev_mem_range_handle) - prev_mem_range_handle->next = new_mem_range_handle; - else - global_data.mem_range_head = new_mem_range_handle; - new_mem_range_handle->next = cur_mem_range_handle; - - found_mem_range = new_mem_range_handle; - break; - } - else if(cur_mem_range_handle->size == size) - { - found_mem_range = cur_mem_range_handle; - mem_reg_info = malloc(sizeof(mem_reg_t)); - if(mem_reg_info== NULL) - { - triton_mutex_unlock(&global_data.mutex); - return TRITON_ERR_NOMEM; - } - mem_reg_info->publish_count = 0; - break; - } - } - prev_mem_range_handle = cur_mem_range_handle; - cur_mem_range_handle = cur_mem_range_handle->next; - } - - if(found_mem_range != NULL) - { - global_data.remaining_mem_size -= size; - found_mem_range->mem_reg_info = mem_reg_info; - mem_reg_info->base_address = (void*)((uint64_t)global_data.one_sided_memory + - found_mem_range->offset); - mem_reg_info->user_address = base; /*base is NULL if this function is called from buffer_allocate. - */ - mem_reg_info->size = size; - *handle = mem_reg_info; - triton_mutex_unlock(&global_data.mutex); - return TRITON_SUCCESS; - } - triton_mutex_unlock(&global_data.mutex); - return TRITON_ERR_NOMEM; -} - - -static void compact_free_contiguous_mem_ranges() -{ - mem_range_t* cur_mem_range_handle = global_data.mem_range_head; - mem_range_t* prev_mem_range_handle = NULL; - mem_range_t* temp_mem_range_handle = NULL; - - while(cur_mem_range_handle) - { - if(prev_mem_range_handle && prev_mem_range_handle->mem_reg_info == NULL - &&cur_mem_range_handle->mem_reg_info == NULL) - { - prev_mem_range_handle->next = cur_mem_range_handle->next; - prev_mem_range_handle->size += cur_mem_range_handle->size; - temp_mem_range_handle = cur_mem_range_handle; - cur_mem_range_handle = cur_mem_range_handle->next; - free(temp_mem_range_handle); - continue; - } - prev_mem_range_handle = cur_mem_range_handle; - cur_mem_range_handle = cur_mem_range_handle->next; - } -} - -static triton_ret_t unregister_memory_internal(mem_reg_t* mem_reg_info) -{ - mem_range_t* cur_mem_range_handle = global_data.mem_range_head; - mem_range_t* prev_mem_range_handle = NULL; - - if(!mem_reg_info) - return TRITON_ERR_INVAL; - - if(mem_reg_info->publish_count != 0) - return TRITON_ERR_MEM_IN_USE; - - triton_mutex_lock(&global_data.mutex); - if(mem_reg_info->base_address == mem_reg_info->user_address) - { - /*The handle is asociated to an address created through buffer_allocate. - We cant' destroy it until buffer_free is called on the associated address. - Though, we have to make it impossible to call unregister on it again - */ - mem_reg_info->user_address = NULL; - return TRITON_SUCCESS; - } - while(cur_mem_range_handle) - { - if(cur_mem_range_handle->mem_reg_info == mem_reg_info) - { - global_data.remaining_mem_size += mem_reg_info->size; - free(cur_mem_range_handle->mem_reg_info); - cur_mem_range_handle->mem_reg_info = NULL; - compact_free_contiguous_mem_ranges(); - triton_mutex_unlock(&global_data.mutex); - return TRITON_SUCCESS; - } - prev_mem_range_handle = cur_mem_range_handle; - cur_mem_range_handle = cur_mem_range_handle->next; - } - triton_mutex_unlock(&global_data.mutex); - - return TRITON_ERR_INVAL; //The handle is not a valid registered memory handle. -} - -void* buffer_allocate(size_t size) -{ - mem_reg_t* handle = NULL; - triton_ret_t ret = register_memory_internal(NULL, (uint64_t)size, &handle); - if(ret != TRITON_SUCCESS) - return NULL; - return handle->base_address; -} - -void buffer_free(void* buffer) -{ - mem_range_t *range = find_associated_mem_range(buffer); - triton_assert(range); - if(range) - { - if(range->mem_reg_info->base_address == range->mem_reg_info->user_address) - { - triton_err(triton_log_default, - "%s:%d: Attempt to free a memory that has not yet been unpublished", __FILE__, __LINE__); - return; - } - triton_ret_t ret = unregister_memory_internal(range->mem_reg_info); - triton_assert(ret == TRITON_SUCCESS); - } - else - { - triton_assert(0); - /*trying to free a memory region that has not been allocated with this mechanism! - Shouldn't we trigger a kind of cataclismic event ... just like the stdlib.h free would do. - In debug mode, the asssert will trigger; but in release mode, if we're silent, - the programmer might think that everything is ok while he is leaking memory (buffer will - not be duly freed through the appropriate mechanism if the programmer thinks that THIS mechanism - is already doing it. - */ - } -} - -/** -* Register memory for the communication subsystem. -*/ -triton_ret_t register_memory(void* base, //the base address of the memory region - size_t size, // the size in bytes of the memory region - triton_mem_reg_handle_t* handle /*handle returned by - the memory registration*/ - ) -{ - if(size == 0) - { - *handle = ZERO_SIZE_MEM_HANDLE; - return TRITON_SUCCESS; - } - return register_memory_internal(base, (uint64_t)size, (mem_reg_t**)(handle)); -} - -/** -* Unregister memory. -*/ -triton_ret_t unregister_memory(triton_mem_reg_handle_t handle /*handle previously returned - by the memory registration*/ - ) -{ - if(handle == ZERO_SIZE_MEM_HANDLE) - return TRITON_SUCCESS; - return unregister_memory_internal((mem_reg_t*)(((void*)(handle)))); -} - -static int is_valid_mem_reg_info(mem_reg_t* mem_reg_info) -{ - mem_range_t* cur_mem_range_handle = global_data.mem_range_head; - - while(cur_mem_range_handle) - { - if(cur_mem_range_handle->mem_reg_info == mem_reg_info) - return 1; - cur_mem_range_handle = cur_mem_range_handle->next; - } - return 0; -} - -/** -* Expose a registered memory portion to remote peers. -*/ -__blocking triton_ret_t publish_memory(triton_mem_reg_handle_t in_handle, //handle of a registered memory - size_t offset, /*offset (in bytes) from the base address - where the memory publishing should start from*/ - size_t size, //size (in bytes) of the published area - triton_mem_pub_handle_t* out_handle /*handle returned from - publishing the memory*/ - ) -{ - if(size == 0) - { - *out_handle = ZERO_SIZE_MEM_HANDLE; - return TRITON_SUCCESS; - } - - mem_reg_t* mem_reg_info = (mem_reg_t*)in_handle; - mem_pub_t* mem_pub_info = NULL; - - if(!is_valid_mem_reg_info(mem_reg_info )) - return TRITON_ERR_INVAL; - - if(mem_reg_info->size < (uint64_t)offset || (uint64_t)size > mem_reg_info->size - (uint64_t)offset) - return TRITON_ERR_MEM_OUT_OF_RANGE; - - mem_pub_info = (mem_pub_t*)calloc(1, sizeof(mem_pub_t)); - if(!mem_pub_info) - return TRITON_ERR_NOMEM; - - mem_pub_info->offset = offset + ((uint64_t)mem_reg_info->base_address - - (uint64_t)global_data.one_sided_memory); - mem_pub_info->size = size; - mem_pub_info->mem_reg_info = mem_reg_info; - mem_pub_info->client_rank = global_data.rank; - - triton_mutex_lock(&global_data.mutex); - mem_reg_info->publish_count++; - triton_mutex_unlock(&global_data.mutex); - - //if the user-address has not been allocated using buffer_allocate - if(mem_reg_info->base_address != mem_reg_info->user_address) - { - memcpy((void*)((uint64_t)global_data.one_sided_memory + mem_pub_info->offset), - (void*)((uint64_t)mem_reg_info->user_address + mem_pub_info->offset), - mem_pub_info->size); - } - *out_handle = (uint64_t)mem_pub_info; - - return TRITON_SUCCESS; -} - -/** -* End access permissions to a memory portion previously exposed. This is meant to be a client-side operation -*/ -__blocking triton_ret_t unpublish_memory(triton_mem_pub_handle_t handle /*handle previously - returned by a memory publishing*/ - ) -{ - if(handle == ZERO_SIZE_MEM_HANDLE) - return TRITON_SUCCESS; - - mem_pub_t* mem_pub_info = (mem_pub_t*)handle; - mem_reg_t* mem_reg_info = mem_pub_info->mem_reg_info; - - if(!is_valid_mem_reg_info(mem_reg_info )) - return TRITON_ERR_INVAL; - if(mem_pub_info->made_from_deserialization) - return TRITON_ERR_INVAL; //this handle is not allowed to unpublish stuff - - //if the user-address has not been allocated using buffer_allocate - if(mem_reg_info->base_address != mem_reg_info->user_address) - { - memcpy((void*)((uint64_t)mem_reg_info->user_address + mem_pub_info->offset), - (void*)((uint64_t)global_data.one_sided_memory + mem_pub_info->offset), - mem_pub_info->size); - } - - free(mem_pub_info); - - triton_mutex_lock(&global_data.mutex); - mem_reg_info->publish_count--; - triton_mutex_unlock(&global_data.mutex); - - return TRITON_SUCCESS; -} - - -int get_serialized_handle_size(serialized_handle_type_t handle_type) -{ - switch(handle_type) - { - case MEM_PUB_HANDLE_TYPE: - return (int)sizeof(mem_pub_t); - case WAIT_HANDLE_TYPE: - return (int) sizeof(triton_wait_handle_t); - default: - triton_assert("unknown serialized_handle_type_t"); - } - return -1; //this should never happen -} - - - - -triton_ret_t get_serialized_handle(serialized_handle_type_t handle_type, - void* pointer_to_in_handle, - char* serialized_handle) -{ - mem_pub_t* mem_pub = NULL; - switch(handle_type) - { - case MEM_PUB_HANDLE_TYPE: - mem_pub = *((mem_pub_t**)pointer_to_in_handle); - memcpy(serialized_handle, mem_pub, sizeof(mem_pub_t)); - break; - case WAIT_HANDLE_TYPE: - memcpy(serialized_handle, *((triton_wait_handle_t**)pointer_to_in_handle), - sizeof(triton_wait_handle_t)); - break; - default: - return TRITON_ERR_INVAL; - } - return TRITON_SUCCESS; -} - -triton_ret_t set_serialized_handle(serialized_handle_type_t handle_type, - void* pointer_to_out_handle, - char* serialized_handle) -{ - mem_pub_t* mem_pub = NULL; - switch(handle_type) - { - case MEM_PUB_HANDLE_TYPE: - mem_pub = (mem_pub_t*)calloc(1, sizeof(mem_pub_t)); - if(!mem_pub) - return TRITON_ERR_NOMEM; - memcpy(mem_pub, serialized_handle, sizeof(mem_pub_t)); - mem_pub->made_from_deserialization = 1; - *((triton_mem_pub_handle_t*)pointer_to_out_handle) = (triton_mem_pub_handle_t)mem_pub; - break; - case WAIT_HANDLE_TYPE: - memcpy(pointer_to_out_handle, serialized_handle, sizeof(triton_wait_handle_t)); - break; - default: - return TRITON_ERR_INVAL; - } - return TRITON_SUCCESS; -} - -triton_ret_t free_serialized_handle(serialized_handle_type_t handle_type, - void* pointer_to_in_handle) -{ - mem_pub_t* mem_pub = NULL; - switch(handle_type) - { - case MEM_PUB_HANDLE_TYPE: - mem_pub = *((mem_pub_t**)pointer_to_in_handle); - if(!mem_pub->made_from_deserialization) - return TRITON_ERR_INVAL; - free(mem_pub); - *((triton_mem_pub_handle_t*)pointer_to_in_handle) = 0; - break; - case WAIT_HANDLE_TYPE: - *((triton_wait_handle_t*)pointer_to_in_handle) = 0; - break; - default: - return TRITON_ERR_INVAL; - } - return TRITON_SUCCESS; - -} - -__blocking triton_ret_t get(triton_msg_method_addr_t from, - triton_mem_pub_handle_t handle, - triton_iov_item_t* iovs, - char** dest_bufs, - int iov_items_count) -{ - int ret; - MPI_Datatype origin_hind_type; //TODO: CREATE THIS VIA dest_bufs AND MPI_Get_address - MPI_Datatype target_hind_type; - int* lengths = NULL; - MPI_Aint* indices = NULL; - int i; - - if(iov_items_count == 0) - return TRITON_SUCCESS; - if(iov_items_count == 1) - { - ret = MPI_Get(dest_bufs[0], iovs[0].size, MPI_CHAR, (int)from, iovs[0].offset, - iovs[0].size, MPI_CHAR, global_data.win); - RETURN_ON_MPI_FAILURE(ret, "MPI_Get failed"); - return TRITON_SUCCESS; - } - - lengths = malloc(iov_items_count*sizeof(int)); - if(!lengths) return TRITON_ERR_NOMEM; - indices = malloc(iov_items_count*sizeof(MPI_Aint)); - if(!indices) - { - free(lengths); - return TRITON_ERR_NOMEM; - } - for(i=0; i<iov_items_count; i++) - { - lengths[i]=(int)iovs[i].size; - indices[i]=(MPI_Aint)iovs[i].offset; - } - ret = MPI_Type_create_hindexed(iov_items_count, lengths, indices, MPI_CHAR, &target_hind_type); - if(ret != MPI_SUCCESS) - { - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Type_create_hindexed"); - } - ret = MPI_Type_commit(&target_hind_type); - if(ret != MPI_SUCCESS) - { - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Type_commit"); - } - - for(i=0; i<iov_items_count; i++) - MPI_Get_address(dest_bufs[i], &indices[i]); - ret = MPI_Type_create_hindexed(iov_items_count, lengths, indices, MPI_CHAR, &origin_hind_type); - if(ret != MPI_SUCCESS) - { - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Type_create_hindexed"); - } - ret = MPI_Type_commit(&origin_hind_type); - if(ret != MPI_SUCCESS) - { - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Type_commit"); - } - - ret = MPI_Get(MPI_BOTTOM, 1, origin_hind_type, (int)from, 0, - 1, target_hind_type, global_data.win); - - MPI_Type_free(&target_hind_type); - MPI_Type_free(&origin_hind_type); - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Get failed"); - return TRITON_SUCCESS; -} - -__blocking triton_ret_t put(triton_msg_method_addr_t to, - triton_mem_pub_handle_t handle, - triton_iov_item_t* iovs, - char** src_bufs, - int iov_items_count) -{ - int ret; - MPI_Datatype origin_hind_type; //TODO: CREATE THIS VIA dest_bufs AND MPI_Get_address - MPI_Datatype target_hind_type; - int* lengths = NULL; - MPI_Aint* indices = NULL; - int i; - - if(iov_items_count == 0) - return TRITON_SUCCESS; - if(iov_items_count == 1) - { - ret = MPI_Put(src_bufs[0], iovs[0].size, MPI_CHAR, (int)to, iovs[0].offset, - iovs[0].size, MPI_CHAR, global_data.win); - RETURN_ON_MPI_FAILURE(ret, "MPI_Get failed"); - return TRITON_SUCCESS; - } - - lengths = malloc(iov_items_count*sizeof(int)); - if(!lengths) return TRITON_ERR_NOMEM; - indices = malloc(iov_items_count*sizeof(MPI_Aint)); - if(!indices) - { - free(lengths); - return TRITON_ERR_NOMEM; - } - for(i=0; i<iov_items_count; i++) - { - lengths[i]=(int)iovs[i].size; - indices[i]=(MPI_Aint)iovs[i].offset; - } - ret = MPI_Type_create_hindexed(iov_items_count, lengths, indices, MPI_CHAR, &target_hind_type); - if(ret != MPI_SUCCESS) - { - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Type_create_hindexed"); - } - ret = MPI_Type_commit(&target_hind_type); - if(ret != MPI_SUCCESS) - { - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Type_commit"); - } - - for(i=0; i<iov_items_count; i++) - MPI_Get_address(src_bufs[i], &indices[i]); - ret = MPI_Type_create_hindexed(iov_items_count, lengths, indices, MPI_CHAR, &origin_hind_type); - if(ret != MPI_SUCCESS) - { - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Type_create_hindexed"); - } - ret = MPI_Type_commit(&origin_hind_type); - if(ret != MPI_SUCCESS) - { - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Type_commit"); - } - - ret = MPI_Put(MPI_BOTTOM, 1, origin_hind_type, (int)to, 0, - 1, target_hind_type, global_data.win); - - MPI_Type_free(&target_hind_type); - MPI_Type_free(&origin_hind_type); - free(lengths); - free(indices); - RETURN_ON_MPI_FAILURE(ret, "MPI_Get failed"); - return TRITON_SUCCESS; -} - -__blocking triton_ret_t start_epoch(triton_mem_pub_handle_t handle) //server-side -{ - mem_pub_t *mem_pub = (mem_pub_t*)handle; - int ret = MPI_Win_lock(MPI_LOCK_SHARED, mem_pub->client_rank, 0, global_data.win); - RETURN_ON_MPI_FAILURE(ret, ""); - return TRITON_SUCCESS; -} - - -__blocking triton_ret_t end_epoch(triton_mem_pub_handle_t handle) //server-side -{ - mem_pub_t *mem_pub = (mem_pub_t*)handle; - int ret; - ret = MPI_Win_unlock(mem_pub->client_rank, global_data.win); - RETURN_ON_MPI_FAILURE(ret, ""); - - return TRITON_SUCCESS; -} - - -__blocking triton_ret_t wait(triton_mem_pub_handle_t mem_handle, - triton_wait_handle_t wait_handle) -{ - //SCRATCHING HEAD ... - return TRITON_SUCCESS; -} - -static struct triton_msg_method triton_msg_mpi_method = -{ - .name = "mpi", - - .addr_self = mpi_addr_self, - .addr_equal = mpi_addr_equal, - .addr_to_string = mpi_addr_to_string, - .addr_lookup = mpi_addr_lookup, - .addr_free = mpi_addr_free, - .addr_copy = mpi_addr_copy, - .new_tag = mpi_new_tag, - - .group_add = mpi_group_add, - .send = mpi_send, - .recv = mpi_recv, - .recv_any = mpi_recv_any, - - .set_one_sided_threshold = set_one_sided_threshold, - .get_one_sided_threshold = get_one_sided_threshold, - .buffer_allocate = buffer_allocate, - .buffer_free = buffer_free, - .register_memory = register_memory, - .unregister_memory = unregister_memory, - .publish_memory = publish_memory, - .unpublish_memory = unpublish_memory, - .get_serialized_handle_size = get_serialized_handle_size, - .set_serialized_handle = set_serialized_handle, - .get_serialized_handle = get_serialized_handle, - .free_serialized_handle = free_serialized_handle, - .get = get, - .put = put, - .start_epoch = start_epoch, - .end_epoch = end_epoch, - .wait = wait -}; - - -static __blocking triton_ret_t perform_common_client_server_init() -{ - int ret; - ret = MPI_Comm_rank(MPI_COMM_WORLD, &global_data.rank); - RETURN_ON_MPI_FAILURE(ret, "MPI_Comm_rank failed"); - - //Create the Win for one-sided. - triton_mutex_init(&global_data.mutex, NULL); - global_data.one_sided_memory = malloc(TRITON_MPI_REGISTERED_RMA_MEM_SIZE ); - if(global_data.one_sided_memory == NULL) - return TRITON_ERR_NOMEM; - global_data.remaining_mem_size = TRITON_MPI_MAX_USABLE_REGISTERED_MEM_SIZE; - global_data.mem_range_head = (mem_range_t*)calloc(1, sizeof(mem_range_t)); - if(global_data.mem_range_head == NULL) - { - free(global_data.one_sided_memory); - return TRITON_ERR_NOMEM; - } - global_data.mem_range_head->offset = 0; - global_data.mem_range_head->size = TRITON_MPI_MAX_USABLE_REGISTERED_MEM_SIZE; - global_data.mem_range_head->mem_reg_info = NULL; - - ret = MPI_Win_create(global_data.one_sided_memory, TRITON_MPI_REGISTERED_RMA_MEM_SIZE, - sizeof(char), MPI_INFO_NULL, MPI_COMM_WORLD, &global_data.win); - if(ret != MPI_SUCCESS) - { - free(global_data.one_sided_memory); - RETURN_ON_MPI_FAILURE(ret, "Failed to create MPI_Win for one-sided"); - } - return TRITON_SUCCESS; -} - -inline triton_ret_t triton_mpi_init_client() -{ - return perform_common_client_server_init(); -} - -triton_ret_t triton_msg_mpi_init(void) -{ - triton_ret_t tret; - tret = perform_common_client_server_init(); - if(tret != TRITON_SUCCESS) - return tret; - - assert(group_table == NULL); - group_table = triton_hash_init(mpi_group_compare, triton_hash_32bit_hash, 1024); - if(!group_table) - { - triton_err(triton_log_default, "%s:%d: Failed to initialize group hashtable", __FILE__, __LINE__); - assert(group_table); - } - -#if 0 //JZ - tret = triton_msg_method_register(&triton_msg_mpi_method); -#endif //JZ - return TRITON_SUCCESS; -} - -void perform_common_client_server_finalize() -{ - MPI_Win_free(&global_data.win); - free(global_data.one_sided_memory); - triton_assert(global_data.mem_range_head->next == NULL); - free(global_data.mem_range_head); - triton_mutex_destroy(&global_data.mutex); -} - -inline void triton_mpi_finalize_client() -{ - perform_common_client_server_finalize(); -} - -void triton_msg_mpi_finalize(void) -{ -#if 0 //JZ - triton_msg_method_unregister("mpi"); - - if(group_table != NULL) - { - triton_hash_destroy_and_finalize(group_table, struct triton_mpi_group, link, group_entry_free); - group_table = NULL; - } -#endif //JZ - perform_common_client_server_finalize(); -} - -void triton_msg_mpi_init_register(void); - -void triton_msg_mpi_init_register(void) -{ - triton_init_register("triton.net.mpi", triton_msg_mpi_init, triton_msg_mpi_finalize, - NULL, "triton.net.message", "triton.resource.mpi"); -} - -#ifdef __JZ -struct triton_msg_method* get_msg_method(){return &triton_msg_mpi_method;} -#endif //__JZ - - -#if 1 -char large_buf[512*MB]; -void test_mem_reg_pub() -{ - triton_ret_t ret; - triton_mem_reg_handle_t reg_handles[10]; - triton_mem_pub_handle_t pub_handles[10]; - char* mems[10]; - - int i = 0, j = 0; - - - ret = register_memory(&large_buf[j], 10, &reg_handles[i]); //0 - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 10); - UNIT_TEST(global_data.mem_range_head->size == 10); - UNIT_TEST(global_data.mem_range_head->next != NULL); - UNIT_TEST(global_data.mem_range_head->next->size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 10); - UNIT_TEST(global_data.mem_range_head->next->next == NULL); - - ret = unregister_memory(reg_handles[i]); - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE); - UNIT_TEST(global_data.mem_range_head->size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE); - UNIT_TEST(global_data.mem_range_head->next == NULL); - - ret = register_memory(&large_buf[j], 10, &reg_handles[i++]); //0 - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 10); - UNIT_TEST(global_data.mem_range_head->size == 10); - UNIT_TEST(global_data.mem_range_head->next != NULL); - UNIT_TEST(global_data.mem_range_head->next->size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 10); - UNIT_TEST(global_data.mem_range_head->next->next == NULL); - - j+=1024; - ret = register_memory(&large_buf[j], 10*MB, &reg_handles[i++]); //1 - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 10 - 10*MB); - - j+=1024; - ret = register_memory(&large_buf[j], 10*MB, &reg_handles[i++]);//2 - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 10 - 20*MB); - - ret = register_memory(&large_buf[j], 500*MB, &reg_handles[i++]);//3 - UNIT_TEST(ret == TRITON_ERR_NOMEM); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 10 - 20*MB); - - j+=1024; - ret = register_memory(&large_buf[j], 2*MB, &reg_handles[i++]);//4 - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 10 - 22*MB); - - j+=1024; - ret = register_memory(&large_buf[j], 400*MB, &reg_handles[i++]);//5 - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 10 - 422*MB); - - ret = unregister_memory(reg_handles[5]); - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 10 - 22*MB); - - ret = unregister_memory(reg_handles[0]); - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 22*MB); - - ret = unregister_memory(reg_handles[2]); - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 12*MB); - - ret = unregister_memory(reg_handles[1]); - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 2*MB); - - ret = unregister_memory(reg_handles[1]); //should fail - UNIT_TEST(ret == TRITON_ERR_INVAL); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 2*MB); - - ret = unregister_memory(reg_handles[3]); //should fail - UNIT_TEST(ret == TRITON_ERR_INVAL); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 2*MB); - - ret = unregister_memory(reg_handles[4]); - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE); - UNIT_TEST(global_data.mem_range_head->next == NULL); - UNIT_TEST(global_data.mem_range_head->size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE); - - /*--------------------------------------------*/ - - mems[0] = buffer_allocate(128*MB); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 128*MB); - UNIT_TEST(global_data.mem_range_head->size == 128*MB); - UNIT_TEST(global_data.mem_range_head->next != NULL); - UNIT_TEST(global_data.mem_range_head->next->size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 128*MB); - UNIT_TEST(global_data.mem_range_head->next->next == NULL); - - ret = register_memory(&large_buf[j], 128*MB, &reg_handles[0]); - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 256*MB); - UNIT_TEST(global_data.mem_range_head->size == 128*MB); - UNIT_TEST(global_data.mem_range_head->next != NULL); - UNIT_TEST(global_data.mem_range_head->next->size == 128*MB); - UNIT_TEST(global_data.mem_range_head->next->next != NULL); - UNIT_TEST(global_data.mem_range_head->next->next->size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 256*MB); - UNIT_TEST(global_data.mem_range_head->next->next->next == NULL); - - mems[1] = buffer_allocate(128*MB); - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 384*MB); - - ret = register_memory(&large_buf[j], 128*MB, &reg_handles[2]); - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == 0); - - ret = publish_memory(reg_handles[0], 120*MB, 9*MB, &pub_handles[0]); //should fail - UNIT_TEST(ret == TRITON_ERR_MEM_OUT_OF_RANGE); - UNIT_TEST(global_data.remaining_mem_size == 0); - UNIT_TEST(((mem_reg_t*)reg_handles[0])->publish_count == 0); - - ret = publish_memory(reg_handles[0], 128*MB + 1, 9*MB, &pub_handles[0]); //should fail - UNIT_TEST(ret == TRITON_ERR_MEM_OUT_OF_RANGE); - UNIT_TEST(global_data.remaining_mem_size == 0); - UNIT_TEST(((mem_reg_t*)reg_handles[1])->publish_count == 0); - - ret = publish_memory(reg_handles[2], 120*MB + 1, 7*MB, &pub_handles[1]); // - /*free(g_m); - free((client_side_mem_pub_t*)pub_handles[1]);*/ - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == 0); - UNIT_TEST(((mem_reg_t*)reg_handles[2])->publish_count == 1); - - ret = publish_memory(reg_handles[2], 120*MB + 1, 7*MB, &pub_handles[2]); // - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == 0); - UNIT_TEST(((mem_reg_t*)reg_handles[2])->publish_count == 2); - - ret = publish_memory(reg_handles[4], 0 + 1, 7*MB, &pub_handles[3]); // - UNIT_TEST(ret == TRITON_ERR_INVAL); - UNIT_TEST(global_data.remaining_mem_size == 0); - - /*--------------------*/ - - ret = unregister_memory(reg_handles[0]); //should work - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 384*MB); - - ret = unregister_memory(reg_handles[2]); //should fail - UNIT_TEST(ret == TRITON_ERR_MEM_IN_USE); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 384*MB); - - ret = unpublish_memory(pub_handles[1]); - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 384*MB); - UNIT_TEST(((mem_reg_t*)reg_handles[2])->publish_count == 1); - - ret = unregister_memory(reg_handles[0]); //should fail - UNIT_TEST(ret == TRITON_ERR_INVAL); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 384*MB); - - ret = unpublish_memory(pub_handles[2]); - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 384*MB); - UNIT_TEST(((mem_reg_t*)reg_handles[2])->publish_count == 0); - - ret = register_memory(mems[1], 128*MB, &reg_handles[9]); - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 384*MB); - - buffer_free(mems[1]); //buffer_free should be prohibited internally - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 384*MB); - - unregister_memory(reg_handles[9]); - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 384*MB); - - buffer_free(mems[1]); //buffer_free should be allowed now - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 256*MB); - - ret = unregister_memory(reg_handles[2]); //should work - UNIT_TEST(ret == TRITON_SUCCESS); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE - 128*MB); - - buffer_free(mems[0]); - UNIT_TEST(global_data.remaining_mem_size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE); - UNIT_TEST(global_data.mem_range_head->size == TRITON_MPI_REGISTERED_RMA_MEM_SIZE); - UNIT_TEST(global_data.mem_range_head->next == NULL); -} -#endif diff --git a/code/src/net/mpi/one_sided_interface_mpiimpl_2/test.c b/code/src/net/mpi/one_sided_interface_mpiimpl_2/test.c deleted file mode 100644 index 30c9d14..0000000 --- a/code/src/net/mpi/one_sided_interface_mpiimpl_2/test.c +++ /dev/null @@ -1,810 +0,0 @@ -#include "mock.h" -#include <mpi.h> -#include <stdio.h> -#include <errno.h> -#include "triton-message-method.h" -#include <time.h> -#include <stdlib.h> -#include <string.h> -#include <stdarg.h> - -#define ABORT_ON_MPI_FAILURE(ret,err_msg_format,...) \ -do \ -{ \ - if((ret) != MPI_SUCCESS) \ - { \ - triton_err(triton_log_default, "%s:%d: [MPI Proc rank %d]" err_msg_format, \ - __FILE__, __LINE__, rank, ##__VA_ARGS__); \ - MPI_Abort(MPI_COMM_WORLD, ret); \ - } \ -}while(0) - -#define KB 1024 -#define MB 1048576 - -#define MAX_NB_SERVERS 255 /* For I'm using byte offsets and writing/reading char to check the test results. - I'm writing ranks to know who wrote where and who read from where; so - more than 255 is troublesome. This is just for the readability of this test. - */ - -#define MAX_DATA_SIZE 50*MB -int rank = -1; -int size = -1; -int server_ranks[MAX_NB_SERVERS]; -int nb_servers = -1; - -struct triton_msg_method *tmm = NULL; - -typedef struct node_link_t -{ - struct node_link_t* next; -}node_link_t; - -void add_node(void** node_head, void** node_tail, void* node) -{ - node_link_t **head = (node_link_t**)(node_head); - node_link_t **tail = (node_link_t**)(node_tail); - if(*tail) - { - (*tail)->next = (node_link_t*)node; - } - else - { - *head = (node_link_t*)node; - } - *tail = (node_link_t*)node; -} - -void remove_node(void** node_head, void** node_tail, void* node) -{ - node_link_t *cur = (node_link_t*)(*node_head), *previous = NULL; - while(cur) - { - if(cur == node) - { - if(previous) - previous->next = cur->next; - else - *node_head = cur->next; - if(*node_tail == cur) - *node_tail = previous; - break; - } - previous = cur; - cur = cur->next; - } -} - -int is_prime(int n) -{ - int i, half = n/2; - if(n == 0) - return 0; - if(n == 1) - return 1; - for(i=2; i<=half; i++) - if( (n%i) == 0) - return 0; - return 1; -} - -int is_local_process_a_server(rank, size) -{ - if(size <=2 ) - return rank >=size -1; - return is_prime(rank); -} - -void discover_servers(void) -{ - int ret; - int i; - nb_servers = 0; - int is_server = is_local_process_a_server(rank, size); - ret = MPI_Allgather(&is_server, 1, MPI_INT, server_ranks, 1, MPI_INT, MPI_COMM_WORLD); - if(ret!=MPI_SUCCESS) - MPI_Abort(MPI_COMM_WORLD, ret); - - /*At this point, global_data.server_ranks is more like a server map with 1 at indices(ranks) which - are servers and 0 at others. We want to have a contiguous array having the ranks of the servers. - */ - for(i=0; i<size; i++) - { - if(server_ranks[i]) - server_ranks[nb_servers++] = i; - } -} - -FILE* trace_file = NULL; -#define TRACE_IN_FILE(...) do{fprintf(trace_file, __VA_ARGS__);}while(0) - -void init_trace_in_file() -{ - char file_name[100]; - sprintf(file_name, "trace_%s_rank_%d", is_local_process_a_server(rank, size)?"server":"client", - rank); - trace_file = fopen(file_name, "w"); -} - -void close_trace_in_file() -{ - fclose(trace_file); -} - -char* server_data = NULL; -char* read_server_data = NULL; -char* write_server_data = NULL; -triton_mem_reg_handle_t server_mrh; - -void init_server_data() -{ - triton_ret_t tret; - if(!server_data) - { - server_data = tmm->buffer_allocate(MAX_DATA_SIZE); - triton_assert(server_data); - tret = tmm->register_memory(server_data, MAX_DATA_SIZE, &server_mrh); - triton_assert(tret == TRITON_SUCCESS); - } - memset(server_data, rank, MAX_DATA_SIZE); - write_server_data = read_server_data = server_data; - //write_server_data = &server_data[SERVER_DATA_SIZE/2]; -} - -void free_server_data() -{ - triton_ret_t tret; - if(server_data) - { - tret = tmm->unregister_memory(server_mrh); - triton_assert(tret == TRITON_SUCCESS); - tmm->buffer_free(server_data); - } -} -static inline int get_nb_responding_servers(triton_iov_item_t* iov, int *iov_indices, int nb_iov_items) -{ - return MIN(nb_servers, nb_iov_items); -} - - -/* -concerned_servers is assumed to be able to contain as many ranks as there are servers in -the job. -Returns the number of iov the server is expected to service -*/ -int get_response_map_for_specific_server(int nb_iov_items, int* iov_indices, - int server_rank) -{ - /*start at rank of primarily contacted server to ensure that it gets at least 1*/ - - int i, k; - int next_server_index = -1; - for(i=0; i<nb_servers; i++) - if(server_ranks[i] == rank) - { - next_server_index = i; - break; - } - triton_assert(next_server_index != -1); - for(i=0, k=0; i<nb_iov_items; i++) - { - if(server_ranks[next_server_index] == server_rank) - iov_indices[k++] = i; - next_server_index = (next_server_index+1)%nb_servers; - } - return k; -} - -#define TEST_TAG_READ 0 -#define TEST_TAG_WRITE 1 -#define TEST_TAG_ACKS 2 -#define TEST_TAG_READ_SCHEDULE_TRANSFER_REQUEST 4 -#define TEST_TAG_WRITE_SCHEDULE_TRANSFER_REQUEST 6 -#define TEST_TAG_ACK_FRAGMENT 7 - -#define TEST_TAG_SHUTDOWN 10 - -#define MAX_RPC_SIZE 4*KB -#define MAX_NB_IOV_ITEMS 50 - -typedef enum io_type_t -{ - IOT_READ, - IOT_WRITE -}io_type_t; - - -void notify_for_client_shutdown() -{ - int i; - char dummy; - for(i=0; i<nb_servers; i++) - MPI_Send(&dummy, 1, MPI_CHAR, server_ranks[i], TEST_TAG_SHUTDOWN, MPI_COMM_WORLD); -} - -/*out_mem_pub_params contains the offset and size of the required mem_pub_handle -*/ -void get_io_size(triton_iov_item_t* iovs, int nb_iov_items, triton_iov_item_t* out_mem_pub_params) -{ - size_t smallest_start = iovs[0].offset, temp_start, largest_end = iovs[0].offset + iovs[0].size, temp_end; - int i; - for(i=1; i<nb_iov_items; i++) - { - temp_start = iovs[i].offset; - temp_end = temp_start + iovs[i].size; - smallest_start = MIN(smallest_start, temp_start); - largest_end = MAX(largest_end, temp_end); - } - out_mem_pub_params->offset = smallest_start; - out_mem_pub_params->size = largest_end - smallest_start; -} - - -/* -rank is the rank of the peer which created the mem_pub_handle -unique_token is used to identify a specific request. Two pairs (rank, unique_token) -will never exist at the same time -*/ -int build_io_rpc(triton_iov_item_t* iovs, int* valid_iov_indices, int nb_iov_items, - char* serialized_mem_pub, int serialized_mem_pub_size, int rank, uint64_t unique_token, char* rpc_packet) -{ - triton_iov_item_t partial_iov[MAX_NB_IOV_ITEMS]; - int i; - for(i=0; i<nb_iov_items; i++) - { - partial_iov[i].offset = iovs[valid_iov_indices[i]].offset; - partial_iov[i].size = iovs[valid_iov_indices[i]].size; - } - int rpc_size = 0; - memcpy(rpc_packet, &rank, sizeof(rank)); - rpc_size += sizeof(rank); - - memcpy(rpc_packet + rpc_size, &unique_token, sizeof(uint64_t)); - rpc_size+=sizeof(uint64_t); - - memcpy(rpc_packet + rpc_size, serialized_mem_pub, serialized_mem_pub_size); - rpc_size += serialized_mem_pub_size; - - memcpy(rpc_packet + rpc_size, &nb_iov_items, sizeof(nb_iov_items)); - rpc_size += sizeof(nb_iov_items); - - memcpy(rpc_packet + rpc_size, partial_iov, sizeof(triton_iov_item_t)*nb_iov_items); - rpc_size += sizeof(triton_iov_item_t)*nb_iov_items; - - return rpc_size; -} - -void extract_io_rpc_data(char* rpc_packet, triton_iov_item_t** iovs, int *nb_iov_items, - char** serialized_mem_pub, int *client_rank, uint64_t *unique_token) -{ - *client_rank = *((int*)rpc_packet); - *unique_token = *(uint64_t*)(rpc_packet + sizeof(int)); - *serialized_mem_pub = rpc_packet + sizeof(int) + sizeof(uint64_t); - *nb_iov_items = *((int*)(*serialized_mem_pub+tmm->get_serialized_handle_size(MEM_PUB_HANDLE_TYPE))); - *iovs = (triton_iov_item_t*)(*serialized_mem_pub+tmm->get_serialized_handle_size(MEM_PUB_HANDLE_TYPE) - +sizeof(int)); -} - - -#if 0 -typedef struct mem_range_t -{ - struct mem_range_t* next; - void* mem_reg_info; - uint64_t offset; - uint64_t size; - int made_from_deserialization; //1 if made from deserialization - int client_rank; //the rank fo the peer that initially published the memory -}mem_range_t; -#endif - -/*Assumes that memory was registered upfront*/ -__blocking void do_client_io(char* mem_start, triton_mem_reg_handle_t mrh, triton_iov_item_t* iovs, - int nb_iov_items, io_type_t iot) -{ - triton_iov_item_t mem_pub_handle_params; - int primary_server_rank = server_ranks[rand()%nb_servers]; - int ret; - int i; - triton_ret_t tret; - triton_mem_pub_handle_t mph; - MPI_Status status; - char some_buf[MAX_RPC_SIZE]; - char rpc_packet[MAX_RPC_SIZE]; //we're not really doing RPC here but this packet will contain approximately - //the same data as the equivalent RPC payload - int serialized_mem_pub_h_size; - triton_wait_handle_t wait_handle; - int rpc_size = 0; - get_io_size(iovs, nb_iov_items, &mem_pub_handle_params); - - int *valid_iov_indices = malloc(nb_iov_items*sizeof(int)); - for(i=0; i<nb_iov_items; i++) - valid_iov_indices[i] = i; - - do - { - tret = tmm->publish_memory(mrh, mem_pub_handle_params.offset, mem_pub_handle_params.size, &mph); - if(tret != TRITON_SUCCESS) - JZ_WAIT_FOR_DEBUGGER(); - else - break; - //triton_assert(tret == TRITON_SUCCESS); - } - while(1); - - serialized_mem_pub_h_size = tmm->get_serialized_handle_size(MEM_PUB_HANDLE_TYPE); - tret = tmm->get_serialized_handle(MEM_PUB_HANDLE_TYPE, (void*)&mph, some_buf); - triton_assert(tret == TRITON_SUCCESS); - - rpc_size = build_io_rpc(iovs, valid_iov_indices, nb_iov_items, some_buf, serialized_mem_pub_h_size, - rank, (uint64_t)mph, rpc_packet); - free(valid_iov_indices); - - ret = MPI_Send(rpc_packet, rpc_size, MPI_CHAR, primary_server_rank, - iot == IOT_READ ? TEST_TAG_READ : TEST_TAG_WRITE, MPI_COMM_WORLD); - ABORT_ON_MPI_FAILURE(ret, "MPI_Send failed"); - - ret = MPI_Recv(rpc_packet, rpc_size, MPI_CHAR, primary_server_rank, - TEST_TAG_ACKS, - MPI_COMM_WORLD, &status); - ABORT_ON_MPI_FAILURE(ret, "MPI_Recv failed"); - - tret = tmm->set_serialized_handle(WAIT_HANDLE_TYPE, (void*)(&wait_handle), - rpc_packet); - triton_assert(tret == TRITON_SUCCESS); - tret = tmm->wait(mph, wait_handle); - triton_assert(tret == TRITON_SUCCESS); - tret = tmm->free_serialized_handle(WAIT_HANDLE_TYPE, (void*)&wait_handle); - triton_assert(tret == TRITON_SUCCESS); - tret = tmm->unpublish_memory(mph); - triton_assert(tret == TRITON_SUCCESS); - - if(iot == IOT_READ) - { - int i; - //Display read data. - printf("[CLIENT Rank %d]READ DATA: ", rank); - for(i=mem_pub_handle_params.offset; i<mem_pub_handle_params.size+mem_pub_handle_params.offset; i++) - TRACE_IN_FILE("%d: %d\n", i, mem_start[i]); - TRACE_IN_FILE("\n\n"); - } - else - TRACE_IN_FILE("[Client rank %d] DONE WRITING STUFF: ", rank); - - TRACE_IN_FILE("\n"); -} - -char buffer[MAX_DATA_SIZE]; -__blocking void enter_client_loop() -{ - triton_ret_t tret; - uint64_t io_span_per_client = 10000; - uint64_t initial_offset = rank*io_span_per_client; - uint64_t write_offset = 5000; - triton_mem_reg_handle_t mrh1; - triton_iov_item_t read_iovs[] = {{initial_offset+5, 5}, {initial_offset+12, 8}, - {initial_offset+40, 10}, {initial_offset+100, 10}, {initial_offset+120, 10}, - {initial_offset+150, 10}, {initial_offset+512, 128}, - {initial_offset+768, 20}, {initial_offset+2*KB, KB}}; - initial_offset += write_offset; - triton_iov_item_t write_iovs[] = {{initial_offset+5, 5}, {initial_offset+12, 8}, - {initial_offset+40, 10}, {initial_offset+100, 10}, {initial_offset+120, 10}, - {initial_offset+150, 10}, {initial_offset+512, 128}, - {initial_offset+768, 20}, {initial_offset+2*KB, KB}}; - memset(buffer, rank, MAX_DATA_SIZE); - if(1) - { - tret = tmm->register_memory(buffer, MB, &mrh1); - triton_assert(tret == TRITON_SUCCESS); - do_client_io(buffer, mrh1, write_iovs, 6, IOT_WRITE); - - do_client_io(buffer, mrh1, read_iovs, 6, IOT_READ); - - tret = tmm->unregister_memory(mrh1); - triton_assert(tret == TRITON_SUCCESS); - } - - notify_for_client_shutdown(); -} - -void display_server_side_read_data(char* buf_start, triton_iov_item_t *iov, int nb_iov_items, int client_rank) -{ - int i; - triton_iov_item_t data_span; - get_io_size(iov, nb_iov_items, &data_span); - TRACE_IN_FILE("<-----TRACING MEMORY AFTER I/O WITH CLIENT %d------->\n", client_rank); - for(i=data_span.offset; i<data_span.offset+data_span.size; i++) - { - TRACE_IN_FILE("%d:%d\n", i, buf_start[i]); - } - TRACE_IN_FILE("</-----TRACING MEMORY AFTER I/O WITH CLIENT %d------->\n\n", client_rank); -} - -void do_server_io(int client_rank, triton_mem_pub_handle_t mem_pub_handle, triton_iov_item_t *iov, - int nb_iov_items, int test_type) -{ - int i; - triton_ret_t tret; - char* buf_start = test_type == TEST_TAG_READ? read_server_data: write_server_data; - char **bufs = malloc(sizeof(char*)*nb_iov_items); - triton_assert(bufs); - for(i=0; i<nb_iov_items; i++) - bufs[i] = (char*)(buf_start + iov[i].offset); - if(test_type == TEST_TAG_READ) - { - tret = tmm->put((triton_msg_method_addr_t)client_rank, - mem_pub_handle, - iov, bufs, nb_iov_items); - triton_assert(tret == TRITON_SUCCESS); - } - else - { - tret = tmm->get((triton_msg_method_addr_t)client_rank, - mem_pub_handle, - iov, bufs, nb_iov_items); - triton_assert(tret == TRITON_SUCCESS); - } - triton_assert(tret == TRITON_SUCCESS); - free(bufs); -} - - -typedef struct server_side_wait_info_t -{ - struct server_side_wait_info_t *next; - int client_rank; - int nb_expected_acks_from_other_servers; - uint64_t unique_token; -}server_side_wait_info_t; - - -server_side_wait_info_t* wait_info_head = NULL; -server_side_wait_info_t* wait_info_tail = NULL; - - -static void add_wait_info(int client_rank, int nb_expected_acks_from_other_servers, uint64_t unique_token) -{ - server_side_wait_info_t* wait_info = malloc(sizeof(server_side_wait_info_t)); - triton_assert(wait_info); - wait_info->client_rank = client_rank; - wait_info->next = NULL; - wait_info->nb_expected_acks_from_other_servers = nb_expected_acks_from_other_servers; - wait_info->unique_token = unique_token; - add_node((void**)(&wait_info_head), (void**)(&wait_info_tail), (void*)wait_info); -} - -static server_side_wait_info_t* get_wait_info(uint64_t unique_token) -{ - server_side_wait_info_t* wait_info = wait_info_head; - while(wait_info) - { - if(wait_info->unique_token == unique_token) - return wait_info; - wait_info = wait_info->next; - } - return NULL; -} - -static inline void remove_wait_info(server_side_wait_info_t* wait_info) -{ - remove_node((void**)(&wait_info_head), (void**)(&wait_info_tail), (void*)wait_info); - free(wait_info); -} - -__blocking void enter_server_loop() -{ - int nb_client_shutdowns = 0; - int clear_to_read_next_incoming_rpc = 1; - triton_mutex_t mutex; - int serialized_mem_pub_h_size = tmm->get_serialized_handle_size(MEM_PUB_HANDLE_TYPE); - char listener_rpc_packet[MAX_RPC_SIZE]; - MPI_Status listener_status; - - triton_mutex_init(&mutex, NULL); //to protect the shutdown variable and possibly other stuff - pwait - { - pprivate char rpc_packet[MAX_RPC_SIZE]; - pprivate triton_iov_item_t* iovs; - pprivate char* serialized_mem_pub; - pprivate int client_rank; - pprivate int rpc_size; - pprivate uint64_t unique_token; - - pprivate int iov_indices[MAX_NB_IOV_ITEMS]; - pprivate char io_rpc_packets[MAX_NB_SERVERS][MAX_RPC_SIZE]; - pprivate MPI_Status status; - pprivate int ret; - pprivate triton_ret_t tret; - pprivate triton_mem_pub_handle_t mem_pub_handle; - pprivate int nb_scheduled_servers = 0; - pprivate int i; - pprivate int j; - pprivate int nb_iov_items; - pprivate int nb_iov_items_for_specific_server; - pprivate MPI_Request *requests = NULL; - pprivate MPI_Status *statuses = NULL; - pprivate server_side_wait_info_t* wait_info; - triton_iov_item_t partial_iov[MAX_NB_IOV_ITEMS]; - - for(;;) - { - if(nb_client_shutdowns == size - nb_servers) - { - JZ_PRINT("[rank %d], Server is shutting down. Hasta luego!\n", rank); - break; - } - while(!clear_to_read_next_incoming_rpc); //might use sleep or a semaphore, etc. instead here ... - ret = MPI_Recv(listener_rpc_packet, MAX_RPC_SIZE, MPI_CHAR, - MPI_ANY_SOURCE, MPI_ANY_TAG, MPI_COMM_WORLD, &listener_status); - clear_to_read_next_incoming_rpc = 0; - pbranch - { - memcpy(rpc_packet, listener_rpc_packet, MAX_RPC_SIZE); - memcpy(&status, &listener_status, sizeof(MPI_Status)); - clear_to_read_next_incoming_rpc = 1; - - switch(status.MPI_TAG) - { - case TEST_TAG_READ: - case TEST_TAG_WRITE: - JZ_PRINT("Primary Server %d handling I/O\n", rank); - extract_io_rpc_data(rpc_packet, &iovs, &nb_iov_items, &serialized_mem_pub, - &client_rank, &unique_token); - nb_scheduled_servers = get_nb_responding_servers(iovs, iov_indices, nb_iov_items); - if(nb_scheduled_servers > 1) - { - requests = malloc(sizeof(MPI_Request)*nb_scheduled_servers); - statuses = malloc(sizeof(MPI_Status)*nb_scheduled_servers); - if(!requests ||!statuses) - { - triton_err(triton_log_default, - "%s:%d: Server (rank %d). Memory allocation failed in server internal loop", - __FILE__, __LINE__, (int)rank); - MPI_Abort(MPI_COMM_WORLD, errno); //The best thing to do from here is to abort - } - add_wait_info(client_rank, nb_scheduled_servers - 1, unique_token); - } - - /* - We adopt the following simple response map strategy. Starting from itself, - this primarily contacted server assigns each iov_item to the next server rank (cycling) - until all the iov_items are assigned. - */ - for(i=0, j=0; i<nb_servers; i++) - { - if(server_ranks[i] == rank) - continue; //handle self stuff in last position - nb_iov_items_for_specific_server = get_response_map_for_specific_server( - nb_iov_items, iov_indices, server_ranks[i]); - if(nb_iov_items_for_specific_server == 0) - continue; //the response map does not concern this server_ranks[i] - rpc_size = build_io_rpc(iovs, iov_indices, nb_iov_items_for_specific_server, - serialized_mem_pub, serialized_mem_pub_h_size, client_rank, - unique_token, io_rpc_packets[j]); - - ret = MPI_Isend(io_rpc_packets[j], rpc_size, MPI_CHAR, - server_ranks[i], - status.MPI_TAG == TEST_TAG_READ? - TEST_TAG_READ_SCHEDULE_TRANSFER_REQUEST: - TEST_TAG_WRITE_SCHEDULE_TRANSFER_REQUEST, - MPI_COMM_WORLD, &requests[j]); - j++; - ABORT_ON_MPI_FAILURE(ret, "MPI_Isend in server internal loop"); - } - - /*This server sends its share now*/ - tret = tmm->set_serialized_handle(MEM_PUB_HANDLE_TYPE, (void*)(&mem_pub_handle), - serialized_mem_pub); - triton_assert(tret == TRITON_SUCCESS); - nb_iov_items_for_specific_server = get_response_map_for_specific_server( - nb_iov_items, iov_indices, rank); - for(i=0; i<nb_iov_items_for_specific_server; i++) - { - partial_iov[i].offset = iovs[iov_indices[i]].offset; - partial_iov[i].size = iovs[iov_indices[i]].size; - } - tret = tmm->start_epoch(mem_pub_handle); - triton_assert(tret == TRITON_SUCCESS); - do_server_io(client_rank, mem_pub_handle, partial_iov, nb_iov_items_for_specific_server, - status.MPI_TAG); - tret = tmm->end_epoch(mem_pub_handle); - if(status.MPI_TAG == TEST_TAG_WRITE) - { - display_server_side_read_data(server_data, partial_iov, - nb_iov_items_for_specific_server, client_rank); //TEST - } - triton_assert(tret == TRITON_SUCCESS); - tret = tmm->free_serialized_handle(MEM_PUB_HANDLE_TYPE, (void*)(&mem_pub_handle)); - triton_assert(tret == TRITON_SUCCESS); - if(j) - { - ret = MPI_Waitall(j, requests, statuses); - ABORT_ON_MPI_FAILURE(ret, "MPI_Waitall in server internal loop"); - free(requests); - free(statuses); - } - else - { - ret = MPI_Send(&unique_token, sizeof(uint64_t), MPI_CHAR, client_rank, TEST_TAG_ACKS, - MPI_COMM_WORLD); //For now, what is sent is not important - ABORT_ON_MPI_FAILURE(ret, "MPI_Send in server internal loop"); - } - break; - - case TEST_TAG_READ_SCHEDULE_TRANSFER_REQUEST: - case TEST_TAG_WRITE_SCHEDULE_TRANSFER_REQUEST: - extract_io_rpc_data(rpc_packet, &iovs, &nb_iov_items, &serialized_mem_pub, - &client_rank, &unique_token); - - tret = tmm->set_serialized_handle(MEM_PUB_HANDLE_TYPE, (void*)(&mem_pub_handle), - serialized_mem_pub); - triton_assert(tret == TRITON_SUCCESS); - tret = tmm->start_epoch(mem_pub_handle); - triton_assert(tret == TRITON_SUCCESS); - do_server_io(client_rank, mem_pub_handle, iovs, nb_iov_items, - status.MPI_TAG == TEST_TAG_READ_SCHEDULE_TRANSFER_REQUEST? - TEST_TAG_READ:TEST_TAG_WRITE); - tret = tmm->end_epoch(mem_pub_handle); - if(status.MPI_TAG == TEST_TAG_WRITE_SCHEDULE_TRANSFER_REQUEST) - { - display_server_side_read_data(server_data, iovs, nb_iov_items, client_rank);//TEST - triton_assert(tret == TRITON_SUCCESS); - } - tret = tmm->free_serialized_handle(MEM_PUB_HANDLE_TYPE, (void*)(&mem_pub_handle)); - triton_assert(tret == TRITON_SUCCESS); - ret = MPI_Send(&unique_token, sizeof(uint64_t), MPI_CHAR, status.MPI_SOURCE, - TEST_TAG_ACK_FRAGMENT, MPI_COMM_WORLD); - ABORT_ON_MPI_FAILURE(ret, "MPI_Send in server internal loop"); - break; - - case TEST_TAG_ACK_FRAGMENT: - unique_token = *((uint64_t*)rpc_packet); - triton_mutex_lock(&mutex); - //JZ_WAIT_FOR_DEBUGGER(); - wait_info = get_wait_info(unique_token); - triton_mutex_unlock(&mutex); - triton_assert(wait_info); - if(--wait_info->nb_expected_acks_from_other_servers == 0) - { - ret = MPI_Send(&unique_token, sizeof(uint64_t), MPI_CHAR, wait_info->client_rank, - TEST_TAG_ACKS, MPI_COMM_WORLD); //For now, what is sent is not important - ABORT_ON_MPI_FAILURE(ret, "MPI_Send in server internal loop"); - triton_mutex_lock(&mutex); - remove_wait_info(wait_info); - triton_mutex_unlock(&mutex); - } - break; - - case TEST_TAG_SHUTDOWN: - triton_mutex_lock(&mutex); - nb_client_shutdowns++; - JZ_PRINT("Server %d received client shutdown request no %d\n", rank, nb_client_shutdowns); - triton_mutex_unlock(&mutex); - break; - default: - JZ_PRINT("[Rank %d] What the heck ...\n", rank); - triton_assert(0); - } - } //pbranch - } //for - } //end pwait - triton_mutex_destroy(mutex); -} - -void print_test_description() -{ - if(rank != 0) - return; - printf( - "===============TEST EXPLANATION==========================\n" - "\t*Many of the choices in this test are abitrary! They can be changed by modifying the code\n" - "in test.c:enter_client_loop. Things that are hardcoded(may be \"parameterizable\" in a subsequent \n" - " iteration of this test) are the number, spans and contiguity of the iov. \n" - "Modify them at will; just make sure the registered memory is always\n" - "large enough to hold the whole memory range spanned by an iov\n" - "\t*There is always at least 1 server\n" - "If there are more than 2 processes, all prime ranks are servers\n\n" - "\t*To ease the test output reading, files are used instead of stdout\n" - "At the end of the execution, each process generates a file named trace_{client/server}_rank_{rank}.\n" - "For instance, the process 4 will generate trace_client_rank_4 and the server 3 will generate\n" - "trace_server_rank_3.\n" - "\t*Each process has a reasonnably large I/O buffer that it initializes with its rank.\n" - "Then at the end of the execution, each server writes sections of its buffer where at least\n" - "a client did a write. A server that made no write will generate an empty trace file.\n" - "Clients generate trace content when they make reads.\n" - "The test makes each client read and write once at offsets that are disjoints so that every operation\n" - "could be distinctly traced.\n" - "\t*The chunk of trace below taken from trace_server_rank_1 shows/proves that a read operation\n" - "from the client of rank 9 succeeded. The read was non-contiguous and spans the iov {{5,5},{150,10}}\n" - "in a registered memory which started at offset 95000. As a reminder, iov_items are {offset, size}.\n" - "The ellipsis is inserted for brevity.\n" - "<-----TRACING MEMORY AFTER I/O WITH CLIENT 9------->\n" - "95005:9\n" - "95006:9\n" - "95007:9\n" - "95008:9\n" - "95009:9\n" - "95010:1\n" - "95011:1\n" - "95012:1\n" - "95013:1\n" - "95014:1\n" - "...\n" - "...\n" - "95147:1\n" - "95148:1\n" - "95149:1\n" - "95150:9\n" - "95151:9\n" - "95152:9\n" - "95153:9\n" - "95154:9\n" - "95155:9\n" - "95156:9\n" - "95157:9\n" - "95158:9\n" - "95159:9\n" - "</-----TRACING MEMORY AFTER I/O WITH CLIENT 9------->\n" - "\t*Each client reads and writes 6 iov_tems for each I/O. The primary server is chosen randomly.\n" - "The pipelining strategy simply distributes iov items accross the server in a round-ronbin fashion\n" - "The primary server does that job and always start from itself.\n" - "To ease the trace reading/checking of the pipelining, run a test with 4 processes max so as to have\n" - "a single client (1 non primary number rank which is 0). The program should output the rank of the\n" - "chosen primary server and allow you to determine the response map computation. For instance, with\n" - "4 processes, if rank 2 is selected as primary server, the 6 iovs asked by the client\n" - "(see test.c:enter_client_loop) will be distributed as follows {{5,5},{100,10}}, {{12,8},{120,10}}\n" - "{{40,10},{150,10}} respectively over the ranks 2,3 and 1. Small ranges are chosen just to ease the\n" - "visual assessment in the trace file\n" - "\n\n" - ); - -} - -int main(int argc, char** argv) -{ - int dummy; - srand(time(NULL)); - MPI_Init_thread(&argc, &argv, MPI_THREAD_MULTIPLE, &dummy); - //MPI_Init(&argc, &argv); - MPI_Comm_rank(MPI_COMM_WORLD, &rank); - MPI_Comm_size(MPI_COMM_WORLD, &size); - tmm = get_msg_method(); - assert(tmm); - - if(size == 1) - { - printf("\nHello I'm a jobless server ... who's going to bug me if there is no client, hein?\n" - "\t\tSpawn at least 2 processes. :)\nRun the program with an argument (anything) for help\n\n"); - goto program_end; - } - if(argc > 1) - { - print_test_description(); - goto program_end; - } - if(rank == 0) - printf("\x1b[31mIf help is needed,run the program with an argument (anything)\x1b[0m\n"); - discover_servers(); - init_trace_in_file(); - if(!is_local_process_a_server(rank, size)) - { - triton_mpi_init_client(); - enter_client_loop(); - triton_mpi_finalize_client(); - } - else - { - //JZ_WAIT_FOR_DEBUGGER(); - triton_msg_mpi_init(); - init_server_data(); - enter_server_loop(); - free_server_data(); - triton_msg_mpi_finalize(); - } - close_trace_in_file(); - if(rank == 0) - printf("\x1b[31mIf help is needed, run the program with an argument (anything)\x1b[0m\n"); -program_end: - MPI_Finalize(); - return 0; -} diff --git a/code/src/net/mpi/one_sided_interface_mpiimpl_2/triton-message-method.h b/code/src/net/mpi/one_sided_interface_mpiimpl_2/triton-message-method.h deleted file mode 100644 index 58c82c4..0000000 --- a/code/src/net/mpi/one_sided_interface_mpiimpl_2/triton-message-method.h +++ /dev/null @@ -1,267 +0,0 @@ -#define __JZ -#include "mock.h" - -#ifndef __TRITON_MESSAGE_METHOD_HAE__ -#define __TRITON_MESSAGE_METHOD_HAE__ - -#ifndef __JZ - -#include "src/net/triton-addr.h" -#include "src/net/triton-message.hae" -#endif //__JZ - -typedef uint64_t triton_msg_method_addr_t; -typedef uint32_t triton_msg_method_group_t; - -typedef uint64_t triton_mem_reg_handle_t; - -typedef uint64_t triton_mem_pub_handle_t; - -typedef uint64_t triton_wait_handle_t; - -typedef struct triton_iov_item_t -{ - size_t offset; - size_t size; -}triton_iov_item_t; -/* -See get_serialized_handle below -*/ -typedef enum serialized_handle_type_t -{ - MEM_PUB_HANDLE_TYPE, - WAIT_HANDLE_TYPE /*designate the kind of handle passed to - the wait routine waiting for acks */ -}serialized_handle_type_t; - - -struct triton_msg_method -{ - /** - * The name of this method. Must match the endpoint method prefix. e.g. endpoint=mpi://20, name=mpi - */ - const char *name; - - /** - * Get the local address. This is blocking, because some methods set the "self" address through the hints - * mechanism, which only gets propagated by blocking functions. - */ - __blocking triton_msg_method_addr_t (*addr_self)(void); - - int (*addr_equal)(triton_msg_method_addr_t a1, triton_msg_method_addr_t a2); - - /** - * Get the endpoint string for this address, *without* including the endpoint method prefix. - */ - triton_ret_t (*addr_to_string)(triton_msg_method_addr_t addr, triton_string_t *str); - - /** - * Lookup an address contained in a string. The format depends on the method type, but should - * not include the endpoint method prefix. - */ - triton_ret_t (*addr_lookup)(const char *name, triton_msg_method_addr_t *addr); - - /** - * Free an address - */ - triton_ret_t (*addr_free)(triton_msg_method_addr_t addr); - - /** - * Copy an address - */ - triton_ret_t (*addr_copy)(const triton_msg_method_addr_t orig_addr, triton_msg_method_addr_t *copy); - - /** - * Add a group with the given service name. - */ - triton_ret_t (*group_add)(const char *service_name, triton_msg_method_group_t *group); - - /** - * Get a new tag that can be used for matching sends and receives. - */ - triton_msg_tag_t (*new_tag)(triton_msg_method_group_t group); - - - /** - * Send a message using a particular group and tag - */ - __blocking triton_ret_t (*send)(triton_msg_method_addr_t to, - triton_msg_method_group_t group, - triton_msg_tag_t tag, - int count, - char **buffers, - uint32_t *sizes); - - /** - * Receive (pre-posted) a message from a specific address, on a specific group, with a specific tag. - */ - __blocking triton_ret_t (*recv)(triton_msg_method_addr_t from, - triton_msg_method_group_t group, - triton_msg_tag_t tag, - int count, - char **buffers, - uint32_t *sizes, - uint32_t *bytes_received); - - /** - * Receive a message from a specific group, but from any address with any tag. - */ - __blocking triton_ret_t (*recv_any)(triton_msg_method_group_t group, - triton_msg_method_addr_t *from, - triton_msg_tag_t *tag, - int count, - char **buffers, - uint32_t *sizes, - uint32_t *bytes_received); - - /** - *Set the message size beyond which one-sided communication is used instead 2-sided for message transfers. - * The size is in bytes - */ - triton_ret_t (*set_one_sided_threshold)(size_t size); - - - /** - *Get the message size beyond which one-sided communication is used instead 2-sided for message transfers. - * The size is in bytes - */ - triton_ret_t (*get_one_sided_threshold)(size_t *size); - - /** - * Allocate a buffer for sending and receiving. Method can use this - * call to pin a buffer. - * TODO: Should we include information about the type of buffer to allocate? - */ - void * (*buffer_allocate)(size_t size); - - /** - * Free a buffer allocated with buffer_allocate. - */ - void (*buffer_free)(void *buffer); - - /** - * Register memory for the communication subsystem. - */ - triton_ret_t (*register_memory)(void* base, - size_t size, - triton_mem_reg_handle_t* handle); - /** - * Unregister memory. - */ - triton_ret_t (*unregister_memory)(triton_mem_reg_handle_t handle); - - /** - * Expose a registered memory portion to remote peers. - */ - __blocking triton_ret_t (*publish_memory)(triton_mem_reg_handle_t in_handle, - size_t offset, - size_t size, - triton_mem_pub_handle_t* out_handle); - - /** - * End access permissions to a memory portion previously exposed. - */ - __blocking triton_ret_t (*unpublish_memory)(triton_mem_pub_handle_t handle); - - - /*returns the size of buffer required to get the serialized representation - of the info type passed as argument - */ - int (*get_serialized_handle_size)(serialized_handle_type_t handle_type); - - - /*Ask the net-module to provide the serialized net-module-specific info - associated with the handle *pointer_to_in_handle. - COMMENTS: - 1. pointer_to_in_handle must point to a handle compatible with handle_type - (e.g. if handle_type is MEM_PUB_HANDLE_TYPE, then pointer_to_in_handle must - point to a triton_mem_pub_handle_t). - - 2. serialized_handle must be large enough. A previous call to - get_serialized_handle_size is recommended to determine the required buffer size. - - 3. This function might succeed even if set_serialized_handle (below) was not called previously - to return pointer_to_in_handle. E.g. When publish_memory returns a handle h, any - subsequent call to get_serialized_handle with h and handle_type == MEM_PUB_HANDLE_TYPE - will succeed. - */ - triton_ret_t (*get_serialized_handle)(serialized_handle_type_t handle_type, - void* pointer_to_in_handle, - char* serialized_handle); - - /* - Set an serialized info and get a handle to it. - COMMENTS: - 1. See comment 1 of get_serialized_handle above - - 2. This function both deserializes and sets an serialized data that is maintained for subsequent - references. Example, a valid serialized_handle should subsequently be produced when - get_serialized_handle is invoked with the same handle. - - 3. A call to this function must be coupled with a call to free_serialized_handle below because - serialized data is potentially created. - - 4. Caution: 2 handles to the same serialized info might be different; - handles might not be treated by the net-module like C pointers or even reference-counted pointers. - E.g. two calls to set_serialized_handle that returned h0 and h1 must be coupled with the 2 calls - free_serialized_handle(...,h0) and free_serialized_handle(..., h1) (in any order). So, the 2 calls - free_serialized_handle(...,h1) and free_serialized_handle(..., h1) might be wrong unless something like - free_serialized_handle(...,h1); h1 = h0; free_serialized_handle(..., h1) happens. - */ - triton_ret_t (*set_serialized_handle)(serialized_handle_type_t handle_type, - void* pointer_to_out_handle, - char* serialized_handle); - - /* - Free an serialized info of type handle_type and whose handle is *pointer_to_in_handle. - COMMENTS: - 1. This call will(should) succeed only for handles created through set_serialized_handle. - The function will(should) fail if a handle of the same info type created through a different mechanism - is provided. E.g. This function will(should) fail for any attempt to release the info associated - with a triton_mem_pub_handle_t that was directly created through publish_memory. For such a handle - unpublish_memory is the right way to "free" things. - */ - triton_ret_t (*free_serialized_handle)(serialized_handle_type_t handle_type, - void* pointer_to_in_handle); - - - __blocking triton_ret_t (*get)(triton_msg_method_addr_t from, - triton_mem_pub_handle_t handle, - triton_iov_item_t* iovs, - char** dest_bufs, - int iov_items_count); - - __blocking triton_ret_t (*put)(triton_msg_method_addr_t to, - triton_mem_pub_handle_t handle, - triton_iov_item_t* iovs, - char** src_bufs, - int iov_items_count); - - - __blocking triton_ret_t (*start_epoch)(triton_mem_pub_handle_t in_handle); - - __blocking triton_ret_t (*end_epoch)(triton_mem_pub_handle_t in_handle); - - /* - wait_context is a net-module specific data - */ - __blocking triton_ret_t (*wait)(triton_mem_pub_handle_t handle, void* wait_context); - -}; - -/** - * Methods must register their implementations to the message layer. - */ -triton_ret_t triton_msg_method_register(struct triton_msg_method *method); - -/** - * Unregister a method if its being finalized. - */ -void triton_msg_method_unregister(const char *name); - -/** - * Lookup a registered method based on method name. - */ -struct triton_msg_method* triton_msg_lookup_method(const char* method); - -#endif diff --git a/code/src/net/mpi/tests/module.mk.in b/code/src/net/mpi/tests/module.mk.in deleted file mode 100644 index dc3b16b..0000000 --- a/code/src/net/mpi/tests/module.mk.in +++ /dev/null @@ -1,12 +0,0 @@ -DIR := src/net/mpi/tests - -ifneq (,$(BUILD_MPI)) - -AETESTSRC += $(DIR)/noop-bench.ae - -MODCFLAGS_$(DIR)/noop-bench = $(MPICFLAGS) -MODLDFLAGS_$(DIR)/noop-bench = $(MPILDFLAGS) -MODLIBS_$(DIR)/noop-bench = $(MPILIBS) -MODCC_$(DIR)/noop-bench = $(MPICC) - -endif # BUILD_MPI diff --git a/code/src/net/mpi/tests/noop-bench.ae b/code/src/net/mpi/tests/noop-bench.ae deleted file mode 100644 index b39bb56..0000000 --- a/code/src/net/mpi/tests/noop-bench.ae +++ /dev/null @@ -1,246 +0,0 @@ -#include <sys/types.h> -#include <unistd.h> -#include <limits.h> -#include <stdlib.h> - -#include "src/common/triton-types.h" -#include "src/common/triton-debug.h" -#include <aesop/hints.h> -#include <aesop/aesop.h> -#include "src/remote/encoding.h" -#include "src/remote/remote.hae" -#include "src/remote/retry-method.hae" -#include "src/remote/client-server-exec.hae" -#include "src/net/triton-message.hae" -#include "src/net/triton-message-method.hae" -#include "src/common/resources/scheduling/sched.hae" -#include <aesop/timer.hae> -#include "src/zeroconf/zeroconf.h" -#include "src/net/mpi/mpi-method.h" -#include "src/remote/service.hae" -#include "src/common/triton-init.h" -#include "src/common/triton-bootstrap.hae" -#include "src/state/state.hae" - -/* include server-side remote skeletons */ -#include "src/remote/core_module.h" -/* include client-side remote stubs */ -#include "src/remote/core_stubs.hae" - -#include "mpi.h" - -int nobjects = 0; -int nconcurrent = 0; - -__blocking void do_remote_server(void) -{ - triton_ret_t ret; - - ret = triton_bootstrap_by_nservers(1); - triton_error_assert(ret); - - /* run service engine to process requests */ - assert(AER_DEFAULT_CTX); - aer_service_engine_start(AER_DEFAULT_CTX); - - return; -} - -triton_mutex_t oid_mutex = TRITON_MUTEX_INITIALIZER; - -__blocking void do_remote_noop(void) -{ - triton_ret_t ret; - int32_t result; - uint32_t flags = 0; - const char *server_str = "mpi://0"; - triton_addr_t addr; - uint32_t out; - int started = 0; - - ret = triton_bootstrap_by_nservers(1); - triton_error_assert(ret); - - ret = triton_addr_lookup(server_str, &addr); - triton_error_assert(ret); - - pwait - { - pprivate int i; - - for(i=0; i<nconcurrent; i++) - { - pbranch - { - triton_mutex_lock(&oid_mutex); - while(started < nobjects) - { - started++; - triton_mutex_unlock(&oid_mutex); - - ret = remote_triton_core_noop(AER_DEFAULT_CTX, addr, NULL, NULL); - if(ret != TRITON_SUCCESS) - { - fprintf(stderr, "Error: triton_core_noop() failure.\n"); - } - triton_mutex_lock(&oid_mutex); - } - triton_mutex_unlock(&oid_mutex); - } - } - } - triton_addr_free(addr); -} - -static void usage(void) -{ - fprintf(stderr, - "Usage: noop-bench <obs per proc> <concurrency>\n"); - fprintf(stderr, " (server: rank == 0, clients: rank != 0)\n"); -} - -__blocking int aesop_main(int argc, char **argv) -{ - triton_ret_t ret; - ae_op_id_t op_id; - int rank; - int mpi_ret; - int ae_ret; - MPI_Group orig_group, client_group; - MPI_Comm client_comm; - double test_start, test_end, noop_elapsed; - double noop_elapsed_time, noop_min_elapsed_time; - int scan_ret; - int nclients; - int provided; - int server_rank = 0; - int nprocs; - ae_hints_t hints; - - mpi_ret = MPI_Init_thread(NULL, NULL, MPI_THREAD_MULTIPLE, &provided); - assert(mpi_ret == MPI_SUCCESS); - MPI_Comm_rank(MPI_COMM_WORLD, &rank); - MPI_Comm_size(MPI_COMM_WORLD, &nprocs); - - if(nprocs < 2) - { - fprintf(stderr, "Error: noop-bench requires at least two procs.\n"); - return(-1); - } - - /* parse command line */ - if(argc != 3) - { - usage(); - return(-1); - } - - scan_ret = sscanf(argv[1], "%d", &nobjects); - if(scan_ret != 1 || nobjects < 1) - { - usage(); - return(-1); - } - scan_ret = sscanf(argv[2], "%d", &nconcurrent); - if(scan_ret != 1 || nconcurrent < 1) - { - usage(); - return(-1); - } - - if(rank == 0) - { - /* server */ - ret = retry_method_init(); - triton_error_assert(ret); - ret = aer_service_init(); - triton_error_assert(ret); - ret = aer_remote_register_core(); - triton_error_assert(ret); - } - else - { - /* clients */ - ret = retry_method_init(); - triton_error_assert(ret); - } - ret = triton_state_init(); - triton_error_assert(ret); - - nclients = nprocs-1; - - /* create a communicator for the "clients" */ - mpi_ret = MPI_Comm_group(MPI_COMM_WORLD, &orig_group); - assert(mpi_ret == MPI_SUCCESS); - mpi_ret = MPI_Group_excl(orig_group, 1, &server_rank, &client_group); - assert(mpi_ret == MPI_SUCCESS); - mpi_ret = MPI_Comm_create(MPI_COMM_WORLD, client_group, &client_comm); - assert(mpi_ret == MPI_SUCCESS); - - if(rank == 0) - { - do_remote_server(); - } - else - { - /* clients */ - - /* wait a little bit to make sure the server gets started */ - sleep(3); - - /* barrier and start noop test */ - mpi_ret = MPI_Barrier(client_comm); - assert(mpi_ret == MPI_SUCCESS); - if(rank == 1) - { - printf("# starting noop test.\n"); - } - - test_start = MPI_Wtime(); - do_remote_noop(); - test_end = MPI_Wtime(); - noop_elapsed = test_end - test_start; - - sleep(2); - } - - if(rank >= 1) - { - MPI_Allreduce(&noop_elapsed, &noop_elapsed_time, 1, - MPI_DOUBLE, MPI_MAX, client_comm); - MPI_Allreduce(&noop_elapsed, &noop_min_elapsed_time, 1, - MPI_DOUBLE, MPI_MIN, client_comm); - if(rank == 1) - { - printf("# noop test done.\n"); - - printf("# <nclients> <ops/client> <concurrency> <min> <max> <rate>\n"); - printf("noop\t%d\t%d\t%d\t%f\t%f\t%f\n", - nclients, nobjects, nconcurrent, noop_min_elapsed_time, noop_elapsed_time, (double)nobjects*(double)(nclients)/noop_elapsed_time); - - ret = triton_do_remote_shutdown(); - triton_error_assert(ret); - } - } - - if(rank == 0) - { - aer_service_finalize(); - } - triton_state_finalize(); - retry_method_finalize(); - - MPI_Finalize(); - - return 0; -} -aesop_main_set(aesop_main); - -/* - * Local variables: - * c-indent-level: 4 - * c-basic-offset: 4 - * End: - * - * vim: ts=8 sts=4 sw=4 expandtab - */ diff --git a/code/src/net/ssm/module.mk.in b/code/src/net/ssm/module.mk.in deleted file mode 100644 index 0fd24f7..0000000 --- a/code/src/net/ssm/module.mk.in +++ /dev/null @@ -1,9 +0,0 @@ -DIR := src/net/ssm - -ifneq (,$(BUILD_SSM)) - -LIBSRC += $(DIR)/ssm-tcp-transport.c - -AELIBSRC += $(DIR)/ssm-method.ae - -endif # BUILD_MPI diff --git a/code/src/net/ssm/ssm-method.ae b/code/src/net/ssm/ssm-method.ae deleted file mode 100644 index 42dd9db..0000000 --- a/code/src/net/ssm/ssm-method.ae +++ /dev/null @@ -1,1723 +0,0 @@ -/*Update the semaphores and mutex to use the posix ones if we can reason about the length of the resources they protect and that length is short. In particular, if they are protecting CPU-bound stuff, then, they might be plain sems and mutexes -*/ - -#include "ssm-method.h" -#include "ssm-transport.h" -#include "ssm.h" -#include "ssmptcp.h" - -#include <pthread.h> -#include <getopt.h> -#include <assert.h> -#include <unistd.h> -#include <stdio.h> -#include <limits.h> - -#include <sys/time.h> - - -#include <triton-hash.h> - -#include "src/net/triton-addr.h" -#include "src/net/triton-message.hae" -#include "src/net/triton-message-method.hae" -#include "src/remote/byteswap.h" -#include "src/common/resources/scheduling/sem.hae" - -#include <endian.h> - -#include <semaphore.h> - - -#define JZ_DEBUG -static __attribute__((noinline)) void __jz_wait_for_debugger(const char* file, const char* function, int line, int rank, int once) -{ - static int count = 0; - - if(count && once) - return; - count++; - printf("process ( pid = %d | rank = %d ) is waiting in %s at %s:%d for debugger\n", - getpid(), rank, function, file, line); - fflush(stdout); - for(;;); -} - -#ifdef JZ_DEBUG - #define JZ_WAIT_FOR_DEBUGGER() __jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 0) - #define JZ_WAIT_FOR_DEBUGGER_IF(cond) \ - do{if((cond))__jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 0);}while(0) - #define JZ_WAIT_ONCE_FOR_DEBUGGER() __jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 1) - #define JZ_WAIT_ONCE_FOR_DEBUGGER_IF(cond) \ - do{if((cond))__jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 1);}while(0) -#else - #define JZ_WAIT_FOR_DEBUGGER() - #define JZ_WAIT_FOR_DEBUGGER_IF() - #define JZ_WAIT_ONCE_FOR_DEBUGGER() - #define JZ_WAIT_ONCE_FOR_DEBUGGER_IF() -#endif - - -#define PRINT(...) do{printf(__VA_ARGS__); fflush(stdout);}while(0) -#define TRACE(...) do{PRINT("%s:%d[%s]", __FILE__, __LINE__, __FUNCTION__); PRINT(__VA_ARGS__);}while(0) -#define TRACE_IF(cond,...) do{if(cond)TRACE(__VA_ARGS__);}while(0) - -#define KB 1024 -#define MB 1048576 - -typedef enum {FALSE, TRUE} BOOL; - -static __blocking triton_ret_t triton_sem_count_down_generic (triton_sem_t *sem, int count, BOOL ignore_cancel) -{ - int i; - triton_ret_t tret = TRITON_SUCCESS; - for (i=0; i<count; ++i) - { - tret = triton_sem_down(sem); - if(tret == TRITON_ERR_CANCELED) - { - if(!ignore_cancel) - break; - --i; //make up for the cancellation and wait anyways for the right count - } - } - return tret; -} - -static inline __blocking triton_ret_t triton_sem_count_down (triton_sem_t *sem, int count) -{ - return triton_sem_count_down_generic(sem, count, FALSE); -} - -static inline __blocking triton_ret_t triton_sem_count_down_no_cancel (triton_sem_t *sem, int count) -{ - return triton_sem_count_down_generic(sem, count, TRUE); -} - -#define ENV_ETHERNET_IP "ETHERNET_IP" -#define ENV_ETHERNET_PORT "ETHERNET_PORT" - -#define MAX_RAW_ADDRESS_SIZE 80 - -#define ETHERNET_LOOPBACK "127.0.0.1" - -#define SSM_WAIT_TIMEOUT 2000 //2ms - -/*THE MATCH BITS MANAGEMENT STRATEGY: - 1- The most significant bit distinguishes between 2-sided and one-sided. If set, it is 2-sided - 2- For 2-sided: - The group name is hashed over the 32 next most significant bits - The tags occupy the rest. - */ - - -#define TWO_SIDED_MATCH_BITS_FLAG 0x8000000000000000 //the most significant bit is always set for 2-sided -#define TWO_SIDED_ANY_SOURCE_MATCH_MASK (0x07ffffff) -#define TWO_SIDED_TAG_MASK 0x07ffffffLLU -#define GROUP_MASK (~TWO_SIDED_TAG_MASK) -#define IS_ANY_SOURCE_MATCH_BITS(match_bits) (((match_bits)&TWO_SIDED_ANY_SOURCE_MATCH_MASK)==0) - -#define MAX_ONE_SIDED_MATCH_BITS 0X7ffffffffffffffd -#define PROGRESSION_SHUTDOWN_RESERVED_MATCH_BITS 0x7fffffffffffffff - -#define SSM_MAX_TWO_SIDED_SIZE (4*KB) -#define SSM_MAX_INTERNAL_TWO_SIDED_SIZE (SSM_MAX_TWO_SIDED_SIZE+MAX_RAW_ADDRESS_SIZE) - -#define SSM_ANY_SOURCE 0 - -/* - GENERIC NOTES: - 1- This net-module will potentially manage several transports - 2- TCP ssm_transport is being implemented now. The other transports will definitely be different on a few points. - */ - - -#define PRINT_ERR_MSG(err_msg_format,...) \ - do \ -{ \ - triton_err(triton_log_default, "%s:%d: [pid %d]" err_msg_format, \ - __FILE__, __LINE__, getpid(), ##__VA_ARGS__); \ -}while(0) - -#define ABORT(err_msg_format,...) \ - do \ -{ \ - PRINT_ERR_MSG(err_msg_format,##__VA_ARGS__); \ - exit(1); \ -}while(0) - - -#define ABORT_IF(cond,err_msg_format,...) do{if((cond)) ABORT(err_msg_format,##__VA_ARGS__);}while(0) - - -#define ZERO_SIZE_MEM_HANDLE 0x80000000CAFEBABE - -/*TRANSPORT_TYPE_FROM_ADDR is encoded in the triton_msg_method_addr_t as the most significant byte*/ -#define TRANSPORT_TYPE_MASK_FOR_ADDR 0xff00000000000000 - - -#define TRANSPORT_TYPE_FROM_ADDR(addr) ((((uint64_t)(addr))&TRANSPORT_TYPE_MASK_FOR_ADDR)>>56) - - -static uint64_t ssm_one_sided_size_threshold = SSM_MAX_TWO_SIDED_SIZE; - -static struct triton_hash_table *group_table = NULL; -static const char* transport_type_delimiter = "::"; - -typedef struct network_endpoint_t -{ - ssm_Itp ssm_transport; - ssm_id ssm_instance; - ssm_Iaddr address_interface; - ssm_Haddr local_listener_address_handle; - ssm_me shutdown_me; //the progress engine shutdown is triggered by an ssm_unlink op. - ssm_mr shutdown_mr; - unsigned char raw_address[MAX_RAW_ADDRESS_SIZE]; - ssm_transport_abstraction_t* transport_abstraction; - ssm_transport_type_t transport_type; - - sem_t release_sem; //used in the release phase before shutting down the network endpoint - - triton_list_link_t list_link; - - struct triton_hash_link address_hash_link; //if the address is not encodable over 56-bit, this contains - //the actual address data. It uses the address as a key then - //Transports like TCP do not use this hash_list -}network_endpoint_t; - -network_endpoint_t tcp_network_endpoint; -//TODO [FUTURE TRANSPORTS]: Add network endpoints for other transports when they are available - -network_endpoint_t *default_net_endpoint; - - -triton_msg_method_addr_t default_self_addr; - - -struct -{ - triton_mutex_t mutex; - triton_mutex_t two_sided_mutex; - triton_list_t mem_reg_list; - triton_list_t net_endpoint_list; - triton_list_t generic_recv_data_list; - - uint64_t one_sided_match_bits_counter; //ensure match bit unicity - - triton_list_link_t ptp_match_entries_list; - - pthread_t progress_thread; - - int nb_linked_match_entries; - triton_mutex_t match_entries_mutex; - sem_t match_entries_semaphore; - BOOL exiting; //TRUE when the process is shuting down - -}ssm_global_data; - - -static inline ssm_me triton_ssm_link(ssm_id id, ssm_bits bits, ssm_bits mask, - ssm_pos pos, ssm_me anchor, ssm_cb cb, ssm_Flink flags) -{ - triton_mutex_lock(&ssm_global_data.match_entries_mutex); - ++ssm_global_data.nb_linked_match_entries; - triton_mutex_unlock(&ssm_global_data.match_entries_mutex); - return ssm_link(id, bits, mask, pos, anchor, cb, flags); -} - - -static inline void update_match_entry_count_on_unlink() -{ - triton_mutex_lock(&ssm_global_data.match_entries_mutex); - --ssm_global_data.nb_linked_match_entries; - if(ssm_global_data.exiting && ssm_global_data.nb_linked_match_entries == 0) - sem_post(&ssm_global_data.match_entries_semaphore); - triton_mutex_unlock(&ssm_global_data.match_entries_mutex); -} - -static inline int triton_ssm_unlink(ssm_id id, ssm_me me) -{ - int status = ssm_unlink(id, me); - if(status == SSM_REMOVE_OK) - { - update_match_entry_count_on_unlink(); - } - - return status; -} - -#define FORBID_IF_SHUT_DOWN_IN_PROGRESS() \ - if(ssm_global_data.exiting) return TRITON_ERR_NET_MODULE_SHUTTING_DOWN; - - -static void shutdown_callback_routine(void* cb_args, void* ev_data) -{ - if(((ssm_result)ev_data)->op == SSM_OP_UNLINK) - { - sem_post(&((network_endpoint_t*)cb_args)->release_sem); - update_match_entry_count_on_unlink(); - } -} - -ssm_cb_t shutdown_callback; - -static void init_ethernet() -{ - char* env_val = getenv(ENV_ETHERNET_IP); - char env_val_buf[101]; - unsigned short port; - int i = 0, ip_chunk; - char *token = NULL; - - strcpy(env_val_buf, env_val? env_val:ETHERNET_LOOPBACK); - token = strtok(env_val_buf, "."); - - while(token) - { - ip_chunk = atoi(token); - tcp_network_endpoint.raw_address[i++] = (unsigned char)ip_chunk; - token = strtok(NULL, "."); - ABORT_IF(i>4, "Wrong IPv4 address format specified in %s", ENV_ETHERNET_IP); - } - env_val = getenv(ENV_ETHERNET_PORT); - ABORT_IF(!env_val, "No Ethernet port specified to build local ethernet network endpoint"); - - port = (unsigned short)atoi(env_val); - - memcpy(&tcp_network_endpoint.raw_address[4], &port, sizeof(unsigned short)); - - tcp_network_endpoint.ssm_transport = ssmptcp_new_tp(port, SSM_NOF); - tcp_network_endpoint.ssm_instance = ssm_start(tcp_network_endpoint.ssm_transport, NULL, SSM_NOF); - tcp_network_endpoint.address_interface = ssm_addr(tcp_network_endpoint.ssm_instance); - tcp_network_endpoint.local_listener_address_handle = - tcp_network_endpoint.address_interface->local(tcp_network_endpoint.address_interface); - tcp_network_endpoint.transport_abstraction = get_tcp_abstraction(); - tcp_network_endpoint.transport_type = SSM_TCP; - - shutdown_callback.pcb = shutdown_callback_routine; - shutdown_callback.cbdata = &tcp_network_endpoint; - tcp_network_endpoint.shutdown_me = triton_ssm_link(tcp_network_endpoint.ssm_instance, - PROGRESSION_SHUTDOWN_RESERVED_MATCH_BITS, 0, SSM_POS_HEAD, NULL, &shutdown_callback, - SSM_LINK_AUTO_UNLINK); - tcp_network_endpoint.shutdown_mr = ssm_mr_create(NULL, NULL, 0); - - sem_init(&tcp_network_endpoint.release_sem, 0, 0); - - triton_list_link_clear(&tcp_network_endpoint.list_link); - triton_list_add_back(&tcp_network_endpoint.list_link, &ssm_global_data.net_endpoint_list); -} - -static inline void release_ethernet() -{ - ssm_drop(tcp_network_endpoint.ssm_instance, tcp_network_endpoint.shutdown_me, - tcp_network_endpoint.shutdown_mr); - sem_wait(&tcp_network_endpoint.release_sem); - ssm_stop(tcp_network_endpoint.ssm_instance); - sem_destroy(&tcp_network_endpoint.release_sem); -} - -/*TODO [FUTURE TRANSPORTS]: Provide the init_ethernet and release_ethernet equivalent for - the other transports as well */ - -static inline void set_default_net_endpoint() -{ - default_net_endpoint = &tcp_network_endpoint; -} - - -static ssm_transport_abstraction_t* get_transport_abstraction(ssm_transport_type_t type) -{ - switch(type) - { - case SSM_TCP: - return get_tcp_abstraction(); - default: - ; - } - return NULL; -} - - -static inline ssm_transport_abstraction_t* get_transport_abstraction_from_addr(triton_msg_method_addr_t addr) -{ - return get_transport_abstraction(TRANSPORT_TYPE_FROM_ADDR(addr)); -} - -static network_endpoint_t* get_network_endpoint(ssm_transport_type_t type) -{ - switch(type) - { - case SSM_TCP: - return &tcp_network_endpoint; - - //TODO [FUTURE TRANSPORTS] ... - default: - ; - } - return NULL; -} - -/* -Add a remote address that will be used afterward simply through a triton_msg_method_addr_t -The address is not added if it already exists - This method is used after recv_any gets some address. The address is cached so that - recv could use it afterward. The address is created from raw data; and if it requires - internal data creation, the internal data is cached -*/ -static void add_address(network_endpoint_t* net_endpoint, unsigned char* raw_host_bytes_addr_data, triton_msg_method_addr_t* addr) -{ - switch(net_endpoint->transport_type) - { - case SSM_TCP: - *addr = (triton_msg_method_addr_t)net_endpoint->transport_abstraction->get_address( - raw_host_bytes_addr_data, NULL); - break; - - /*TODO [FUTURE TRANSPORTS]: The other ones will probably have some kind of data structure that will will be created and keyed - with the triton_msg_method_addr_t - */ - default: - ; - } -} - -static __blocking triton_msg_method_addr_t ssm_addr_self(void) -{ - //TODO: How do we know what the caller of this function wants if this ssm_instance of the net-module - //has several transports underneath (E.g. TCP, IB, UDP, some other stuff) - return (triton_msg_method_addr_t)default_net_endpoint->transport_abstraction->get_address( - default_net_endpoint->raw_address, NULL); -} - -static int ssm_addr_equal(triton_msg_method_addr_t a1, triton_msg_method_addr_t a2) -{ - return a1 == a2; -} - -static triton_ret_t ssm_addr_to_string(triton_msg_method_addr_t addr, triton_string_t *addrstr) -{ - char str_addr[MAX_RAW_ADDRESS_SIZE]; - network_endpoint_t* net_endpoint = get_network_endpoint(TRANSPORT_TYPE_FROM_ADDR(addr)); - if(!net_endpoint) - return TRITON_ERR_SSM_TRSPT_TYPE_UNAVAILABLE; - net_endpoint->transport_abstraction->get_str_address((uint64_t)addr, str_addr, net_endpoint->raw_address); - triton_string_init(addrstr, "%s", str_addr); - return TRITON_SUCCESS; -} - -static triton_ret_t ssm_addr_free(triton_msg_method_addr_t addr) -{ - switch(TRANSPORT_TYPE_FROM_ADDR(addr)) - { - case SSM_TCP: - break; //nothing to do for TCP; it uses the type|address enconding - //TODO [FUTURE TRANSPORTS] ... - default: - ; - } - //TODO [FUTURE TRANSPORTS]: The other ones in the future might be reference-counted and freed only when all copies are freed - return TRITON_SUCCESS; -} - -static triton_ret_t ssm_addr_copy(const triton_msg_method_addr_t orig_addr, triton_msg_method_addr_t *copy) -{ - switch(TRANSPORT_TYPE_FROM_ADDR(orig_addr)) - { - case SSM_TCP: - *copy = orig_addr; - break; - //TODO [FUTURE TRANSPORTS]: Other ssm_transport that maintain additional data might need reference counting - default: - ; - } - return TRITON_SUCCESS; -} - -triton_msg_method_addr_t* g_looked_up_addr; - -static triton_ret_t ssm_addr_lookup(const char *name, triton_msg_method_addr_t *addr) -{ - char name_1[251]; - triton_assert(strlen(name)<=250); - if(strncmp(name, "tcp", 3) == 0) - { - strcpy(name_1, name); - *addr = (triton_msg_method_addr_t)tcp_network_endpoint.transport_abstraction->get_address_from_str_addr(name_1, NULL); - g_looked_up_addr = addr; - } //TODO [FUTURE TRANSPORTS] ... - else - { - triton_assert("Unsupported ssm_transport!"); - } - return TRITON_SUCCESS; -} - -ssmptcp_addrargs_t* g_tcp_args; -static ssm_Haddr tcp_get_ssm_addr_from_triton_addr(triton_msg_method_addr_t addr) -{ - unsigned char raw_address[6]; - unsigned char address_arg[SSM_MAX_ADDR_ARG_SIZE]; - ssmptcp_addrargs_t* tcp_args = NULL; - ssm_transport_abstraction_t * tcp_abstraction = get_tcp_abstraction(); - - uint64_t u64_addr = (uint64_t)addr; - - raw_address[0] = (unsigned char)((u64_addr&0xff0000000000)>>40); - raw_address[1] = (unsigned char)((u64_addr&0xff00000000)>>32); - raw_address[2] = (unsigned char)((u64_addr&0xff000000)>>24); - raw_address[3] = (unsigned char)((u64_addr&0xff0000)>>16); - - *((uint16_t*)(&raw_address[4])) = u64_addr&0xffff; - - tcp_abstraction->create_address_arg_data(raw_address, address_arg); - tcp_args = (ssmptcp_addrargs_t*)address_arg; - g_tcp_args = tcp_args; - return tcp_abstraction->create_addr_handle(tcp_network_endpoint.address_interface, tcp_args); -} - -//TODO [FUTURE TRANSPORTS]: ... equivalent of function above - -static triton_ret_t get_ssm_addr_from_triton_addr(triton_msg_method_addr_t addr, ssm_Haddr* the_ssm_addr) -{ - - switch(TRANSPORT_TYPE_FROM_ADDR(addr)) - { - case SSM_TCP: - *the_ssm_addr = tcp_get_ssm_addr_from_triton_addr(addr); - return TRITON_SUCCESS; - //TODO [FUTURE TRANSPORTS] ... - default: - ; - } - return TRITON_ERR_SSM_TRSPT_TYPE_UNAVAILABLE; -} - -#define SSM_TAG_MAX TWO_SIDED_TAG_MASK -struct triton_ssm_group -{ - char *name; - triton_msg_method_group_t group; - int tag_counter; - triton_mutex_t mutex; - struct triton_hash_link link; -}; - - -static triton_ret_t ssm_group_add(const char *service_name, triton_msg_method_group_t *group) -{ - uint32_t h1 = 0, h2 = 0; - struct triton_ssm_group *newgroup; - BOOL group_exists_already = FALSE; - assert(service_name); - assert(group); - - bj_hashlittle2(service_name, strlen(service_name), &h1, &h2); - - triton_mutex_lock(&ssm_global_data.mutex); - if(triton_hash_search(group_table, (triton_msg_method_group_t*)(&h1)) != NULL) - group_exists_already = TRUE; - triton_mutex_unlock(&ssm_global_data.mutex); - if(group_exists_already) - return TRITON_ERR_NAME_HASH_COLLISION; - - newgroup = malloc(sizeof(*newgroup)); - if(newgroup == NULL) return TRITON_ERR_NOMEM; - - newgroup->name = strdup(service_name); - - newgroup->group = h1; - newgroup->tag_counter = 0; - triton_mutex_init(&newgroup->mutex, NULL); - *group = newgroup->group; - - triton_list_link_clear(&newgroup->link); - triton_hash_add(group_table, &newgroup->group, &newgroup->link); - return TRITON_SUCCESS; -} - -static triton_msg_tag_t ssm_new_tag(triton_msg_method_group_t group) -{ - struct triton_hash_link *groupl; - struct triton_ssm_group *group_entry; - triton_msg_tag_t tag; - - /* lookup group */ - groupl = triton_hash_search(group_table, &group); - group_entry = triton_hash_get_entry(groupl, struct triton_ssm_group, link); - triton_mutex_lock(&group_entry->mutex); - ++(group_entry->tag_counter); - if(group_entry->tag_counter >= SSM_TAG_MAX) - { - group_entry->tag_counter = 0; - } - tag = group_entry->tag_counter; - triton_mutex_unlock(&group_entry->mutex); - return tag; -} - - -typedef struct send_comm_info_t -{ - ssm_id ssm_instance; - ssm_tx transaction; - ssm_md md; - ssm_mr mr; - triton_sem_t semaphore; - ssm_status status; - char* intermediate_buffer; -}send_comm_info_t; - - -static void send_callback(void* cb_data, void* event_data) -{ - static int counter = 0; - send_comm_info_t* comm_info = (send_comm_info_t*)cb_data; - ssm_result result = (ssm_result)event_data; - if(result->op == SSM_ST_CANCEL || (result->status == SSM_ST_COMPLETE && result->op == SSM_OP_PUT)) - { - comm_info->status = result->status; - triton_sem_up(&comm_info->semaphore); - } - TRACE("Inside send_callback no = %d (status, op, ssm_bits) = ( %d, %d, 0x%llx)\n", counter++, result->status, result->op, (uint64_t)result->bits); -} - -send_comm_info_t* g_send_comm_info; -ssm_bits g_match_bits; -/*For now, 2-sided operations are limited to a single contiguous buffer*/ -static __blocking triton_ret_t ssm_send( - triton_msg_method_addr_t to, - triton_msg_method_group_t group, - triton_msg_tag_t tag, - int count, - char **buffers, - uint32_t *sizes) -{ - triton_ret_t tret; - int i; - network_endpoint_t* net_endpoint; - send_comm_info_t* comm_info = NULL; - ssm_Haddr the_ssm_addr = NULL; - ssm_cb_t callback; - callback.pcb = send_callback; - - struct triton_hash_link *groupl; - struct triton_ssm_group *group_entry; - ssm_bits match_bits; - - FORBID_IF_SHUT_DOWN_IN_PROGRESS(); - - if(count>1) - return TRITON_ERR_NOT_IMPLEMENTED; //for now, only contiguous stuff is supported - if(sizes[0] > SSM_MAX_TWO_SIDED_SIZE) - return TRITON_ERR_MEM_OUT_OF_RANGE; - - tret = get_ssm_addr_from_triton_addr(to, &the_ssm_addr); - if(tret != TRITON_SUCCESS) - return tret; - net_endpoint = get_network_endpoint(TRANSPORT_TYPE_FROM_ADDR(to)); - triton_assert(net_endpoint); //should not fail if get_ssm_addr_from_triton_addr succeded. - - groupl = triton_hash_search(group_table, &group); - group_entry = triton_hash_get_entry(groupl, struct triton_ssm_group, link); - - /*Most significant bit (msb) is always set. Then we take the 4 msb of the group and finally the tag*/ - match_bits = (TWO_SIDED_MATCH_BITS_FLAG | (((uint64_t)group_entry->group)<<31ULL)|tag); - g_match_bits = match_bits; - - comm_info = malloc(sizeof(send_comm_info_t) + SSM_MAX_INTERNAL_TWO_SIDED_SIZE); - if(!comm_info) - return TRITON_ERR_NOMEM; - - g_send_comm_info = comm_info; - comm_info->intermediate_buffer = ((char*)comm_info + sizeof(send_comm_info_t)); - - /*build and add the sender address; it will be shipped with the data - The first MAX_RAW_ADDRESS_SIZE bytes are reserved for the serialized sender address - */ - net_endpoint->transport_abstraction->get_serialized_address( - net_endpoint->ssm_transport, - net_endpoint->raw_address, (uint8_t*)comm_info->intermediate_buffer); - - memcpy(comm_info->intermediate_buffer + MAX_RAW_ADDRESS_SIZE, buffers[0], sizes[0]); - - comm_info->ssm_instance = net_endpoint->ssm_instance; - callback.cbdata = comm_info; - triton_sem_init(&comm_info->semaphore, 0); - - comm_info->mr = ssm_mr_create(NULL, comm_info->intermediate_buffer, - sizes[0] + MAX_RAW_ADDRESS_SIZE); - - comm_info->md = ssm_md_add(NULL, 0, sizes[0] + MAX_RAW_ADDRESS_SIZE); - comm_info->transaction = ssm_put(net_endpoint->ssm_instance, the_ssm_addr, - comm_info->mr, comm_info->md, match_bits, &callback, SSM_NOF); - - tret = triton_sem_down(&comm_info->semaphore); - - if(tret == TRITON_ERR_CANCELED) - { - ssm_cancel(comm_info->ssm_instance, comm_info->transaction); - triton_sem_count_down_no_cancel(&comm_info->semaphore, 1); - } - else - { - if(comm_info->status != SSM_ST_COMPLETE) - tret = TRITON_ERR_AGAIN; - } - ssm_mr_destroy(comm_info->mr); - ssm_md_release(comm_info->md); - triton_sem_destroy(&comm_info->semaphore); - free(comm_info); - TRACE("Exiting send\n"); - return tret; -} - -/*Each of the data structure below is created for each receive*/ -typedef struct specific_recv_comm_info_t -{ - triton_msg_method_addr_t *sender; //this is a required info for matching - triton_msg_tag_t *tag; - char* buffer; - uint32_t size; - uint32_t* received_bytes; - triton_sem_t semaphore; - triton_list_link_t list_link; -}specific_recv_comm_info_t; - - -typedef struct generic_recv_comm_info_t -{ - network_endpoint_t* net_endpoint; - ssm_me match_entry; - ssm_bits match_bits; - BOOL is_receive_any; - ssm_status status; - triton_list_link_t list_link; - triton_list_t specific_recv_list; -}generic_recv_comm_info_t; - - -static int specific_recv_comm_info_predicate(struct triton_list_link *ll, void *the_key) -{ - triton_msg_method_addr_t key = (triton_msg_method_addr_t)the_key; - specific_recv_comm_info_t* comm_info = triton_list_get_entry(ll, specific_recv_comm_info_t, list_link); - return *comm_info->sender == SSM_ANY_SOURCE || *comm_info->sender == key; -} - -typedef struct generic_recv_comm_key_t -{ - ssm_id ssm_instance; - ssm_bits match_bits; - BOOL is_receive_any; -}generic_recv_comm_key_t; - -static int generic_recv_comm_info_predicate(struct triton_list_link *ll, void *the_key) -{ - generic_recv_comm_key_t* key = (generic_recv_comm_key_t*)the_key; - generic_recv_comm_info_t* comm_info = triton_list_get_entry(ll, generic_recv_comm_info_t, list_link); - if(key->is_receive_any) - return comm_info->is_receive_any && comm_info->net_endpoint->ssm_instance == key->ssm_instance && - (comm_info->match_bits&GROUP_MASK) == (key->match_bits&GROUP_MASK); - return comm_info->net_endpoint->ssm_instance == key->ssm_instance && - comm_info->match_bits == key->match_bits; -} - -/*It is assumed that each call to this function is enclosed in two_sided_mutex -*/ -static specific_recv_comm_info_t* remove_receive(generic_recv_comm_info_t* generic_comm_info, - triton_msg_method_addr_t sender) -{ - TRACE("Inside remove_receive\n"); - specific_recv_comm_info_t* specific_comm_info = NULL; - - triton_list_link_t *ll = triton_list_find(&generic_comm_info->specific_recv_list, - specific_recv_comm_info_predicate, (void*)sender); - if(!ll) - return NULL; - specific_comm_info = triton_list_get_entry(ll, specific_recv_comm_info_t, list_link); - triton_list_del(&specific_comm_info->list_link); - return specific_comm_info; -} - -static inline void cancel_receive(generic_recv_comm_info_t* generic_comm_info,triton_msg_method_addr_t sender) -{ - (void)remove_receive(generic_comm_info, sender); -} - -static void recv_callback(void* cb_data, void* event_data) -{ - static int counter = 0; - TRACE("Entering recv_callback no = %d \n", counter); - int ret; - generic_recv_comm_info_t* comm_info = (generic_recv_comm_info_t*)cb_data; - ssm_result result = (ssm_result)event_data; - ssm_mrinfo_t mr_info; - ssm_mr mr = result->mr; - triton_msg_method_addr_t sender; - specific_recv_comm_info_t* specific_receive_info = NULL; - - triton_assert(result->op == SSM_OP_PUT); - - ret = ssm_mr_getinfo(&mr_info, mr); - triton_assert(ret == 0); - ssm_mr_destroy(mr); - - //extract sender address - comm_info->net_endpoint->transport_abstraction->get_raw_address_in_host_bytes( - comm_info->net_endpoint->ssm_transport, mr_info.base, mr_info.base); //we transform mr_info.base in-place - //so that the address could be referenced afterward with a triton_msg_method_addr_t - add_address(comm_info->net_endpoint, mr_info.base, &sender); - - triton_mutex_lock(&ssm_global_data.two_sided_mutex); - specific_receive_info = remove_receive(comm_info, sender); - if(!specific_receive_info)//probably canceled previously - { - triton_mutex_unlock(&ssm_global_data.two_sided_mutex); - free(mr_info.base); - return; - } - - *specific_receive_info->received_bytes = result->bytes - MAX_RAW_ADDRESS_SIZE; - *specific_receive_info->sender = sender; - *specific_receive_info->tag = result->bits & TWO_SIDED_TAG_MASK; - TRACE("(result->bytes, specific_receive_info->size, specific_receive_info->received_bytes) = (%u, %u, %u)\n", - result->bytes, specific_receive_info->size, *specific_receive_info->received_bytes); - if(TRUE || specific_receive_info->size >= *specific_receive_info->received_bytes) - memcpy(specific_receive_info->buffer, (char*)mr_info.base + MAX_RAW_ADDRESS_SIZE, - 100 /**specific_receive_info->received_bytes*/); - - triton_sem_up(&specific_receive_info->semaphore); - - free(mr_info.base); - - if(comm_info->specific_recv_list.count == 0) - { - triton_ssm_unlink(comm_info->net_endpoint->ssm_instance, comm_info->match_entry); - triton_list_del(&comm_info->list_link); - free(comm_info); - } - triton_mutex_unlock(&ssm_global_data.two_sided_mutex); - TRACE("Exiting recv_callback no = %d (status, op, ssm_bits) = ( %d, %d, 0x%llx)\n", - counter++, result->status, result->op, (uint64_t)result->bits); -} - - -generic_recv_comm_info_t* g_recv_comm; -/* -sender and tag are in/out parameters -*/ -static __blocking triton_ret_t generic_receive(triton_msg_method_group_t group, triton_msg_tag_t *tag, - triton_msg_method_addr_t* sender, char* user_buffer, uint32_t buf_size, uint32_t* received_bytes, - BOOL is_receive_any) -{ - triton_ret_t tret; - - struct triton_hash_link *groupl; - struct triton_ssm_group *group_entry; - generic_recv_comm_key_t key; - triton_list_link_t* ll = NULL; - generic_recv_comm_info_t* comm_info = NULL; - ssm_mr mr; - specific_recv_comm_info_t specific_recv_comm_info; - ssm_cb_t callback; - ssm_bits match_bits; - network_endpoint_t* net_endpoint = NULL; - char *buffer = NULL; - - FORBID_IF_SHUT_DOWN_IN_PROGRESS(); - - specific_recv_comm_info.sender = sender; - specific_recv_comm_info.buffer = user_buffer; - specific_recv_comm_info.size = buf_size; - specific_recv_comm_info.received_bytes = received_bytes; - specific_recv_comm_info.tag = tag; - triton_sem_init(&specific_recv_comm_info.semaphore, 0); - - net_endpoint = is_receive_any ? default_net_endpoint: get_network_endpoint(TRANSPORT_TYPE_FROM_ADDR(*sender)); - triton_assert(net_endpoint); //should not fail if get_ssm_addr_from_triton_addr succeded. - - groupl = triton_hash_search(group_table, &group); - group_entry = triton_hash_get_entry(groupl, struct triton_ssm_group, link); - - match_bits = (TWO_SIDED_MATCH_BITS_FLAG | (((uint64_t)group_entry->group)<<31ULL)); - if(!is_receive_any) - match_bits |= *tag; - - callback.pcb = recv_callback; - buffer = malloc(SSM_MAX_INTERNAL_TWO_SIDED_SIZE); - if(!buffer) - return TRITON_ERR_NOMEM; - - key.ssm_instance = net_endpoint->ssm_instance; - key.match_bits = match_bits; - key.is_receive_any = is_receive_any; - - triton_mutex_lock(&ssm_global_data.two_sided_mutex); - ll = triton_list_find(&ssm_global_data.generic_recv_data_list, generic_recv_comm_info_predicate, &key); - if(ll) - comm_info = triton_list_get_entry(ll, generic_recv_comm_info_t, list_link); - else - { - comm_info = malloc(sizeof(generic_recv_comm_info_t)); - if(!comm_info) - { - triton_mutex_unlock(&ssm_global_data.two_sided_mutex); - triton_sem_destroy(&specific_recv_comm_info.semaphore); - free(buffer); - return TRITON_ERR_NOMEM; - } - comm_info->net_endpoint = net_endpoint; - comm_info->match_bits = match_bits; - triton_list_init(&comm_info->specific_recv_list); - triton_list_link_clear(&comm_info->list_link); - triton_list_add_back(&comm_info->list_link, &ssm_global_data.generic_recv_data_list); - callback.cbdata = comm_info; - comm_info->is_receive_any = is_receive_any; - comm_info->match_entry = triton_ssm_link(net_endpoint->ssm_instance, match_bits, - is_receive_any?TWO_SIDED_ANY_SOURCE_MATCH_MASK:0, SSM_POS_TAIL, NULL, &callback, SSM_NOF); - g_recv_comm = comm_info; - } - - triton_list_link_clear(&specific_recv_comm_info.list_link); - triton_list_add_back(&specific_recv_comm_info.list_link, &comm_info->specific_recv_list); - - mr = ssm_mr_create(NULL, buffer, SSM_MAX_INTERNAL_TWO_SIDED_SIZE); - ssm_post(net_endpoint->ssm_instance, comm_info->match_entry, mr, SSM_NOF); - - triton_mutex_unlock(&ssm_global_data.two_sided_mutex); - - tret = triton_sem_down(&specific_recv_comm_info.semaphore); - TRACE("After triton_sem_down in Generic receive\n"); - if(tret == TRITON_ERR_CANCELED) - { - triton_mutex_lock(&ssm_global_data.two_sided_mutex); - cancel_receive(comm_info, *sender); - triton_mutex_unlock(&ssm_global_data.two_sided_mutex); - triton_sem_destroy(&specific_recv_comm_info.semaphore); - return tret; - } - if(*received_bytes > buf_size) - tret = TRITON_ERR_RECV_TOO_SMALL; - triton_sem_destroy(&specific_recv_comm_info.semaphore); - TRACE("Exiting Generic receive\n"); - return tret; -} - - -static inline __blocking triton_ret_t ssm_recv( - triton_msg_method_addr_t from, - triton_msg_method_group_t group, - triton_msg_tag_t tag, - int count, - char **buffers, - uint32_t *sizes, - uint32_t *bytes_received) -{ - if(count>1) - return TRITON_ERR_NOT_IMPLEMENTED; //for now, only contiguous stuff is supported - return generic_receive(group, &tag, &from, buffers[0], sizes[0], bytes_received, FALSE); -} - - -static inline __blocking triton_ret_t ssm_recv_any( - triton_msg_method_group_t group, - triton_msg_method_addr_t *from, - triton_msg_tag_t *tag, - int count, - char **buffers, - uint32_t *sizes, - uint32_t *bytes_received) -{ - if(count>1) - return TRITON_ERR_NOT_IMPLEMENTED; //for now, only contiguous stuff is supported - *from = SSM_ANY_SOURCE; - return generic_receive(group, tag, from, buffers[0], sizes[0], bytes_received, TRUE); -} - -static int ssm_group_compare(const void *key, struct triton_hash_link *hash_link) -{ - const triton_msg_method_group_t *group1 = (const triton_msg_method_group_t *)key; - struct triton_ssm_group *ssm_group = triton_hash_get_entry(hash_link, struct triton_ssm_group, link); - assert(key); - assert(hash_link); - - return (ssm_group->group == *group1); -} - -static void group_entry_free(void *x) -{ - struct triton_ssm_group *g; - - g = (struct triton_ssm_group *)x; - - triton_mutex_destroy(&g->mutex); - free(g->name); - free(g); -} - -/*---------------One-sided --------------------*/ - -static triton_ret_t set_one_sided_threshold(size_t size) -{ - triton_assert((int64_t)size >= 0); - ssm_one_sided_size_threshold = size; - return TRITON_SUCCESS; -} - - -static triton_ret_t get_one_sided_threshold(size_t *size) -{ - *size = (size_t)ssm_one_sided_size_threshold; - return TRITON_SUCCESS; -} - - -typedef struct mem_reg_t -{ - void* base_address; - uint64_t size; - int publish_count; - BOOL is_registered; //FALSE if this mem_reg_t is only linked to a buffer_allocate'd buffer - BOOL is_internally_allocated; - triton_list_link_t list_link; - ssm_transport_type_t transport_type; //the ssm_transport type this memory has been registered with -}mem_reg_t; - - -typedef struct transaction_t -{ - ssm_tx transaction; - ssm_md md; - ssm_mr mr; - triton_list_link_t list_link; //useful for one-sided -}transaction_t; - -typedef struct mem_pub_t -{ - mem_reg_t* mem_reg_info; - void* base; - uint64_t size; - ssm_mr mr; - ssm_bits match_bits; - ssm_transport_type_t transport_type; - ssm_id transport_instance; - ssm_Haddr initial_owner_address; //This is supposed to be built on the side of the one-sided origin peer - ssm_cb_t callback; - int nb_expected_completions; //for the target - triton_mutex_t mutex; - triton_sem_t comm_semaphore; //MUST be used EXCLUSIVELY for communications. It is not a generic sem - sem_t unlink_semaphore; //MUST be used EXCLUSIVELY for unlink - ssm_me match_entry; - union - { - unsigned char transport_address_data[MAX_RAW_ADDRESS_SIZE]; /*the actual data size depends on transport_type. - And this whole union is expected to be no bigger - than this array - */ - ssmptcp_addrargs_t tcp_args; - //TODO: [FUTURE TRANSPORTS] When they become available, add other ssm_transport address_arg types in this union. - //For now, only TCP is defined - }; - int made_from_deserialization; //1 if made from deserialization - triton_list_t ssm_transaction_list; //transaction list - transaction_t* array_of_tx; //this holds the same transactions in ssm_transaction_list. It is kept to avoid - //myriads of malloc/free calls. See end_epoch -}mem_pub_t; - - -static int mem_reg_predicate(struct triton_list_link *ll, void *base_address) -{ - mem_reg_t* mem_reg = triton_list_get_entry(ll, struct mem_reg_t, list_link); - return mem_reg->base_address == base_address; -} - -static mem_reg_t* find_mem_reg(void* base_address) -{ - mem_reg_t *mem_reg = NULL; - triton_list_link_t* ll = NULL; - if(base_address) - ll = triton_list_find(&ssm_global_data.mem_reg_list, mem_reg_predicate, base_address); - if(ll) - mem_reg = triton_list_get_entry(ll, struct mem_reg_t, list_link); - return mem_reg; -} - - -static triton_ret_t register_memory_internal(void* base, - uint64_t size, - mem_reg_t** handle - ) -{ - mem_reg_t *mem_reg = NULL; - - FORBID_IF_SHUT_DOWN_IN_PROGRESS(); - - mem_reg = find_mem_reg(base); - if(mem_reg) - { - if(mem_reg->is_registered) - return TRITON_ERR_MEM_REGISTERED; - else if(mem_reg->is_internally_allocated) - { - /*The buffer was allocated through the net-module and its handle can just be returned*/ - - mem_reg->is_registered = TRUE; - mem_reg->transport_type = default_net_endpoint->transport_type; - *handle = mem_reg; - return TRITON_SUCCESS; - } - else - triton_assert(0) //this should never happen - } - - mem_reg = malloc(sizeof(mem_reg_t)); - if(!mem_reg) - return TRITON_ERR_NOMEM; - mem_reg->size = size; - mem_reg->publish_count = 0; - if(base) - { - mem_reg->base_address = base; - mem_reg->is_registered = TRUE; - mem_reg->is_internally_allocated = FALSE; - mem_reg->transport_type = default_net_endpoint->transport_type; - } - else //this call is made from buffer_allocate - { - base = malloc(size); - if(!base) - { - triton_list_del(&mem_reg->list_link); - free(mem_reg); - return TRITON_ERR_NOMEM; - } - mem_reg->base_address = base; - mem_reg->is_registered = FALSE; - mem_reg->is_internally_allocated = TRUE; - } - triton_list_link_clear(&mem_reg->list_link); - triton_mutex_lock(&ssm_global_data.mutex); - triton_list_add_back(&mem_reg->list_link, &ssm_global_data.mem_reg_list); - triton_mutex_unlock(&ssm_global_data.mutex); - *handle = mem_reg; - return TRITON_SUCCESS; -} - - -static triton_ret_t unregister_memory_internal(mem_reg_t* mem_reg_info) -{ - triton_list_link_t *ptr, *scratch; - mem_reg_t *mem_reg = NULL; - - triton_list_for_each(ptr, scratch, &ssm_global_data.mem_reg_list) - { - mem_reg = triton_list_get_entry(ptr, struct mem_reg_t, list_link); - if(mem_reg == mem_reg_info) - break; - } - if(!mem_reg) - return TRITON_ERR_INVAL; - if(mem_reg->publish_count != 0) - return TRITON_ERR_MEM_IN_USE; - - if(mem_reg->is_internally_allocated) - { - if(mem_reg->is_registered) //This is really unregister - { - mem_reg->is_registered = FALSE; - return TRITON_SUCCESS; - } - else //this is buffer_free - { - free(mem_reg->base_address); - } - } - triton_mutex_lock(&ssm_global_data.mutex); - triton_list_del(&mem_reg->list_link); - triton_mutex_unlock(&ssm_global_data.mutex); - free(mem_reg); - return TRITON_SUCCESS; -} - -static void* buffer_allocate(size_t size) -{ - mem_reg_t* handle = NULL; - triton_ret_t ret = register_memory_internal(NULL, (uint64_t)size, &handle); - if(ret != TRITON_SUCCESS) - return NULL; - return handle->base_address; -} - -static void buffer_free(void* buffer) -{ - triton_ret_t tret; - mem_reg_t *mem_reg = find_mem_reg(buffer); - if(!mem_reg) - { - PRINT_ERR_MSG("The buffer was not allocated through the net-module"); - return; - } - - if(mem_reg->is_registered) - { - PRINT_ERR_MSG("Trying to free a registered memory"); - return; - } - tret = unregister_memory_internal(mem_reg); - triton_assert(tret); -} - -/** - * Register memory for the communication subsystem. - */ -static triton_ret_t register_memory(void* base, //the base address of the memory region - size_t size, // the size in bytes of the memory region - triton_mem_reg_handle_t* handle /*handle returned by - the memory registration*/ - ) -{ - if(size == 0) - { - *handle = ZERO_SIZE_MEM_HANDLE; - return TRITON_SUCCESS; - } - return register_memory_internal(base, (uint64_t)size, (mem_reg_t**)(handle)); -} - -/** - * Unregister memory. - */ -static triton_ret_t unregister_memory(triton_mem_reg_handle_t handle /*handle previously returned - by the memory registration*/ - ) -{ - if(handle == ZERO_SIZE_MEM_HANDLE) - return TRITON_SUCCESS; - return unregister_memory_internal((mem_reg_t*)(((void*)(handle)))); -} - - -static void target_side_wait_callback(void* callback_args, void* event_data) -{ - ssm_result result = (ssm_result)event_data; - mem_pub_t* mem_pub = (mem_pub_t*)callback_args; - if(result->op == SSM_OP_PUT || result->op == SSM_OP_GET) - triton_sem_up(&mem_pub->comm_semaphore); - else if(result->op == SSM_OP_UNLINK) - { - sem_post(&mem_pub->unlink_semaphore); - update_match_entry_count_on_unlink(); - } -} - - -/** - * Expose a registered memory portion to remote peers. - */ -static __blocking triton_ret_t publish_memory(triton_mem_reg_handle_t in_handle, //handle of a registered memory - size_t offset, /*offset (in bytes) from the base address - where the memory publishing should start from*/ - size_t size, //size (in bytes) of the published area - triton_mem_pub_handle_t* out_handle /*handle returned from - publishing the memory*/ - ) -{ - int ret; - - FORBID_IF_SHUT_DOWN_IN_PROGRESS(); - - if(size == 0) - { - *out_handle = ZERO_SIZE_MEM_HANDLE; - return TRITON_SUCCESS; - } - - mem_reg_t* mem_reg_info = (mem_reg_t*)in_handle; - mem_pub_t* mem_pub = NULL; - - if(!mem_reg_info->is_registered) - return TRITON_ERR_INVAL; - - if(mem_reg_info->size < (uint64_t)offset || - (uint64_t)size > mem_reg_info->size - (uint64_t)offset) - return TRITON_ERR_MEM_OUT_OF_RANGE; - - mem_pub = (mem_pub_t*)calloc(1, sizeof(mem_pub_t)); - if(!mem_pub) - return TRITON_ERR_NOMEM; - - mem_pub->base = (void*)(offset + (uint64_t)mem_reg_info->base_address); - mem_pub->size = size; - mem_pub->mem_reg_info = mem_reg_info; - mem_pub->transport_type = mem_reg_info->transport_type; - mem_pub->transport_instance = default_net_endpoint->ssm_instance; - - triton_mutex_lock(&ssm_global_data.mutex); - mem_reg_info->publish_count++; - mem_pub->match_bits = ssm_global_data.one_sided_match_bits_counter++; - if(ssm_global_data.one_sided_match_bits_counter == MAX_ONE_SIDED_MATCH_BITS) - ssm_global_data.one_sided_match_bits_counter = 0; - triton_mutex_unlock(&ssm_global_data.mutex); - - mem_pub->callback.pcb = target_side_wait_callback; - mem_pub->callback.cbdata = mem_pub; - - mem_pub->mr = ssm_mr_create(NULL, mem_pub->base, mem_pub->size); - mem_pub->match_entry = triton_ssm_link(default_net_endpoint->ssm_instance, mem_pub->match_bits, 0, - SSM_POS_TAIL, NULL, &mem_pub->callback, SSM_LINK_AUTO_UNLINK); - ret = ssm_post(default_net_endpoint->ssm_instance, mem_pub->match_entry, mem_pub->mr, SSM_NOF); - - if(ret) - { - triton_ssm_unlink(mem_pub->transport_instance, mem_pub->match_entry); - free(mem_pub); - return TRITON_ERR_SSM_UNKNOWN; - } - - triton_sem_init(&mem_pub->comm_semaphore, 0); - sem_init(&mem_pub->unlink_semaphore, 0, 0); - - *out_handle = (uint64_t)mem_pub; - - return TRITON_SUCCESS; -} - -/** - * End access permissions to a memory portion previously exposed. - */ -static __blocking triton_ret_t unpublish_memory(triton_mem_pub_handle_t handle /*handle previously - returned by a memory publishing*/ - ) -{ - int status; - if(handle == ZERO_SIZE_MEM_HANDLE) - return TRITON_SUCCESS; - - mem_pub_t* mem_pub = (mem_pub_t*)handle; - mem_reg_t* mem_reg_info = mem_pub->mem_reg_info; - - if(mem_pub->made_from_deserialization) - return TRITON_ERR_INVAL; //this handle is not allowed to be released through unpublish - - status = triton_ssm_unlink(default_net_endpoint->ssm_instance, mem_pub->match_entry); - if(status == SSM_REMOVE_BUSY) - sem_wait(&mem_pub->unlink_semaphore); - triton_assert(status != SSM_REMOVE_INVALID); - - ssm_mr_destroy(mem_pub->mr); - triton_sem_destroy(&mem_pub->comm_semaphore); - sem_destroy(&mem_pub->unlink_semaphore); - free(mem_pub); - - triton_mutex_lock(&ssm_global_data.mutex); - mem_reg_info->publish_count--; - triton_mutex_unlock(&ssm_global_data.mutex); - - return TRITON_SUCCESS; -} - - -static int get_serialized_handle_size(triton_serialized_handle_type_t handle_type, void* handle) -{ - switch(handle_type) - { - case MEM_PUB_HANDLE_TYPE: - return (int)(sizeof(((mem_pub_t*)0)->size) + sizeof(((mem_pub_t*)0)->match_bits) - + default_net_endpoint->transport_abstraction->get_addr_serialization_size()); - default: - triton_assert("unknown triton_serialized_handle_type_t"); - } - return -1; //this should never happen -} - - -static triton_ret_t get_serialized_handle(triton_serialized_handle_type_t handle_type, - void* pointer_to_in_handle, - char* serialized_handle) -{ -/* - void get_serialized_address(ssm_Itp ssm_transport, char* address_data, char* serialization, - char* field_types_in_serialization, int* nb_fields_in_serialization); - */ - - int offset = 0; - - mem_pub_t* mem_pub = NULL; - switch(handle_type) - { - case MEM_PUB_HANDLE_TYPE: - mem_pub = *((mem_pub_t**)pointer_to_in_handle); - serialized_handle[0] = (char)mem_pub->mem_reg_info->transport_type; - offset+=sizeof(char); - *((uint64_t*)(serialized_handle+offset)) = aehton64(mem_pub->size); - offset+=sizeof(uint64_t); - *((int*)(serialized_handle+offset)) = aehton64(mem_pub->match_bits); - offset+=sizeof(int); - default_net_endpoint->transport_abstraction->get_serialized_address( - default_net_endpoint->ssm_transport, - default_net_endpoint->raw_address, (uint8_t*)(serialized_handle+offset)); - break; - default: - return TRITON_ERR_INVAL; - } - return TRITON_SUCCESS; -} - - -static void origin_side_wait_callback(void* callback_args, void* event_data) -{ - ssm_result result = (ssm_result)event_data; - mem_pub_t* mem_pub = (mem_pub_t*)callback_args; - transaction_t *pos, *scratch; - if((result->status == SSM_ST_COMPLETE && (result->op == SSM_OP_PUT || result->op == SSM_OP_GET)) || - result->status == SSM_ST_CANCEL) - { - triton_mutex_lock(&mem_pub->mutex); - triton_list_for_each_entry(pos, scratch, &mem_pub->ssm_transaction_list, transaction_t, list_link) - { - if(pos->transaction == result->tx) - { - triton_list_del(&pos->list_link); - ssm_md_release(pos->md); - ssm_mr_destroy(pos->mr); - triton_sem_up(&mem_pub->comm_semaphore); - break; - } - } - triton_mutex_unlock(&mem_pub->mutex); - } -} - - -static triton_ret_t create_handle_from_serialization(triton_serialized_handle_type_t handle_type, - void* pointer_to_out_handle, - char* serialized_handle) -{ - mem_pub_t* mem_pub = NULL; - ssm_transport_type_t transport_type = serialized_handle[0]; - ssm_transport_abstraction_t* transport_abstraction = get_transport_abstraction(transport_type); - triton_assert(transport_abstraction); - if(!transport_abstraction) - return TRITON_ERR_SSM_TRSPT_TYPE_UNAVAILABLE; - - serialized_handle += sizeof(char); - switch(handle_type) - { - case MEM_PUB_HANDLE_TYPE: - mem_pub = (mem_pub_t*)calloc(1, sizeof(mem_pub_t)); - if(!mem_pub) - return TRITON_ERR_NOMEM; - mem_pub->made_from_deserialization = 1; - serialized_handle += sizeof(char); - mem_pub->size = aentoh64(*((uint64_t*)serialized_handle)); - serialized_handle += sizeof(uint64_t); - mem_pub->match_bits = aentoh32(*((ssm_bits*)serialized_handle)); - serialized_handle += sizeof(ssm_bits); - - mem_pub->callback.pcb = origin_side_wait_callback; - mem_pub->callback.cbdata = mem_pub; - mem_pub->array_of_tx = NULL; - - triton_list_init(&mem_pub->ssm_transaction_list); - - triton_mutex_init(&mem_pub->mutex, NULL); - triton_sem_init(&mem_pub->comm_semaphore, 0); - - switch(transport_type) - { - case SSM_TCP: - transport_abstraction->create_address_arg_data((uint8_t*)serialized_handle, mem_pub->transport_address_data); - break; - default: - ; - } - - *((triton_mem_pub_handle_t*)pointer_to_out_handle) = (triton_mem_pub_handle_t)mem_pub; - break; - default: - return TRITON_ERR_INVAL; - } - return TRITON_SUCCESS; -} - -static triton_ret_t free_serialized_handle(triton_serialized_handle_type_t handle_type, - void* pointer_to_in_handle) -{ - mem_pub_t* mem_pub = NULL; - switch(handle_type) - { - case MEM_PUB_HANDLE_TYPE: - mem_pub = *((mem_pub_t**)pointer_to_in_handle); - - if(!mem_pub->made_from_deserialization) - return TRITON_ERR_INVAL; //it is forbidden to call this function on a handle - //that was not created through serialization - if(mem_pub->array_of_tx) - free(mem_pub->array_of_tx); - - triton_mutex_destroy(&mem_pub->mutex); - triton_sem_destroy(&mem_pub->comm_semaphore); - free(mem_pub); - *((triton_mem_pub_handle_t*)pointer_to_in_handle) = 0; - break; - default: - return TRITON_ERR_INVAL; - } - return TRITON_SUCCESS; -} - - -static void* progress_thread_func(void* args) -{ - network_endpoint_t* net_endpoint; - struct timeval tv; - tv.tv_sec = 0; - tv.tv_usec = SSM_WAIT_TIMEOUT; - while(!ssm_global_data.exiting) - { - if(ssm_global_data.net_endpoint_list.count == 1) //only 1 ssm_instance - ssm_wait(default_net_endpoint->ssm_instance, NULL); - else //several instances - { - triton_list_link_t *ptr, *scratch; - triton_list_for_each(ptr, scratch, &ssm_global_data.net_endpoint_list) - { - net_endpoint = triton_list_get_entry(ptr, network_endpoint_t, list_link); - ssm_wait(net_endpoint->ssm_instance, &tv); - } - } - } - return NULL; -} - -typedef enum one_sided_op_t -{ - TRITON_SSM_OP_GET, - TRITON_SSM_OP_PUT -}one_sided_op_t; - -static triton_ret_t issue_one_sided_op( - triton_mem_pub_handle_t handle, - triton_iov_item_t* iovs, - char** dest_bufs, - int iov_items_count, - one_sided_op_t one_sided_op - ) -{ - int i; - mem_pub_t* mem_pub = (mem_pub_t*)handle; - transaction_t* transactions = NULL; - - FORBID_IF_SHUT_DOWN_IN_PROGRESS(); - - transactions = malloc(iov_items_count*sizeof(transaction_t)); - if(!transactions) - return TRITON_ERR_NOMEM; - - for(i=0; i<iov_items_count; i++) - { - transactions[i].md = ssm_md_add(NULL, iovs[i].offset, iovs[i].size); - transactions[i].mr = ssm_mr_create(NULL, dest_bufs[i], iovs[i].size); - if(one_sided_op == TRITON_SSM_OP_GET) - { - transactions[i].transaction = ssm_get(mem_pub->transport_instance, mem_pub->initial_owner_address, - transactions[i].md, transactions[i].mr, mem_pub->match_bits, &mem_pub->callback, SSM_NOF); - } - else - { - transactions[i].transaction = ssm_put(mem_pub->transport_instance, mem_pub->initial_owner_address, - transactions[i].mr, transactions[i].md, mem_pub->match_bits, &mem_pub->callback, SSM_NOF); - } - triton_list_link_clear(&transactions[i].list_link); - triton_mutex_lock(&mem_pub->mutex); - triton_list_add_back(&transactions[i].list_link, &mem_pub->ssm_transaction_list); - triton_mutex_unlock(&mem_pub->mutex); - mem_pub->array_of_tx = transactions; - } - return TRITON_SUCCESS; -} - -static __blocking triton_ret_t get(triton_msg_method_addr_t from, - triton_mem_pub_handle_t handle, - triton_iov_item_t* iovs, - char** dest_bufs, - int iov_items_count) -{ - return issue_one_sided_op(handle, iovs, dest_bufs, iov_items_count, TRITON_SSM_OP_GET); -} - -static __blocking triton_ret_t put(triton_msg_method_addr_t to, - triton_mem_pub_handle_t handle, - triton_iov_item_t* iovs, - char** src_bufs, - int iov_items_count) -{ - return issue_one_sided_op(handle, iovs, src_bufs, iov_items_count, TRITON_SSM_OP_PUT); -} - - -static __blocking triton_ret_t start_epoch(triton_mem_pub_handle_t handle) -{ - mem_pub_t *mem_pub = (mem_pub_t*)handle; - - FORBID_IF_SHUT_DOWN_IN_PROGRESS(); - - network_endpoint_t *network_endpoint = get_network_endpoint(mem_pub->transport_type); - if(!network_endpoint) - return TRITON_ERR_SSM_TRSPT_TYPE_UNAVAILABLE; - mem_pub->transport_instance = network_endpoint->ssm_instance; - mem_pub->initial_owner_address = network_endpoint->transport_abstraction->create_addr_handle( - network_endpoint->address_interface, (void*)mem_pub->transport_address_data); - return TRITON_SUCCESS; -} - - -static __blocking triton_ret_t end_epoch(triton_mem_pub_handle_t handle) -{ - int i; - triton_ret_t tret; - int count; - mem_pub_t *mem_pub = (mem_pub_t*)handle; - transaction_t *pos, *scratch; - network_endpoint_t *network_endpoint = get_network_endpoint(mem_pub->transport_type); - triton_assert(network_endpoint); - if(!network_endpoint) - { - return TRITON_ERR_SSM_TRSPT_TYPE_UNAVAILABLE; //this should never happen. - } - tret = triton_sem_count_down(&mem_pub->comm_semaphore, mem_pub->ssm_transaction_list.count); - if(tret == TRITON_ERR_CANCELED) - { - triton_mutex_lock(&mem_pub->mutex); - count = mem_pub->ssm_transaction_list.count; - triton_list_for_each_entry(pos, scratch, &mem_pub->ssm_transaction_list, transaction_t, list_link) - { - ssm_cancel(mem_pub->transport_instance, pos->transaction); - } - triton_mutex_unlock(&mem_pub->mutex); - triton_sem_count_down_no_cancel(&mem_pub->comm_semaphore, count); - } - network_endpoint->address_interface->destroy(network_endpoint->address_interface, - mem_pub->initial_owner_address); - return tret; -} - - -static __blocking triton_ret_t wait(triton_mem_pub_handle_t mem_handle, - void* wait_info) -{ - triton_ret_t tret; - mem_pub_t* mem_pub = (mem_pub_t*)mem_handle; - mem_pub->nb_expected_completions = *((int*)wait_info); - return triton_sem_count_down(&mem_pub->comm_semaphore, mem_pub->nb_expected_completions); -} - -static struct triton_msg_method triton_msg_ssm_method = -{ - .name = "ssm", - - .addr_self = ssm_addr_self, - .addr_equal = ssm_addr_equal, - .addr_to_string = ssm_addr_to_string, - .addr_lookup = ssm_addr_lookup, - .addr_free = ssm_addr_free, - .addr_copy = ssm_addr_copy, - .new_tag = ssm_new_tag, - - .group_add = ssm_group_add, - .send = ssm_send, - .recv = ssm_recv, - .recv_any = ssm_recv_any, - - .set_one_sided_threshold = set_one_sided_threshold, - .get_one_sided_threshold = get_one_sided_threshold, - .buffer_allocate = buffer_allocate, - .buffer_free = buffer_free, - .register_memory = register_memory, - .unregister_memory = unregister_memory, - .publish_memory = publish_memory, - .unpublish_memory = unpublish_memory, - .get_serialized_handle_size = get_serialized_handle_size, - .create_handle_from_serialization = create_handle_from_serialization, - .get_serialized_handle = get_serialized_handle, - .free_serialized_handle = free_serialized_handle, - .get = get, - .put = put, - .start_epoch = start_epoch, - .end_epoch = end_epoch, - .wait = wait -}; - - -triton_ret_t triton_msg_ssm_init(void) -{ - triton_ret_t tret; - - triton_mutex_init(&ssm_global_data.mutex, NULL); - triton_mutex_init(&ssm_global_data.two_sided_mutex, NULL); - triton_mutex_init(&ssm_global_data.match_entries_mutex, NULL); - sem_init(&ssm_global_data.match_entries_semaphore, 0, 0); - triton_list_init(&ssm_global_data.mem_reg_list); - triton_list_init(&ssm_global_data.net_endpoint_list); - triton_list_init(&ssm_global_data.generic_recv_data_list); - - ssm_global_data.nb_linked_match_entries = 0; - - triton_sem_init_thread_safe(&ssm_global_data.mutex); - - ssm_global_data.nb_linked_match_entries = 0; - - init_ethernet(); - /*Init the other available tyransports here (e.g. init_ib()*/ - - set_default_net_endpoint(); - - assert(group_table == NULL); - group_table = triton_hash_init(ssm_group_compare, triton_hash_32bit_hash, 1024); - if(!group_table) - { - triton_err(triton_log_default, "%s:%d: Failed to initialize group hashtable", __FILE__, __LINE__); - assert(group_table); - } - - pthread_create(&ssm_global_data.progress_thread, NULL, progress_thread_func, NULL); - tret = triton_msg_method_register(&triton_msg_ssm_method); - return tret; -} - -void triton_msg_ssm_finalize(void) -{ - ssm_global_data.exiting = TRUE; - triton_sem_finalize_thread_safe(&ssm_global_data.mutex); - triton_msg_method_unregister("ssm"); - - if(group_table != NULL) - { - triton_hash_destroy_and_finalize(group_table, struct triton_ssm_group, link, group_entry_free); - group_table = NULL; - } - - pthread_join(ssm_global_data.progress_thread, NULL); - - sem_wait(&ssm_global_data.match_entries_semaphore); //make sure that no match entry is still linked - release_ethernet(); - /*Release the other available transports here (e.g. release_ib()*/ - - triton_mutex_destroy(&ssm_global_data.mutex); - triton_mutex_destroy(&ssm_global_data.two_sided_mutex); - triton_mutex_destroy(&ssm_global_data.match_entries_mutex); - sem_destroy(&ssm_global_data.match_entries_semaphore); -} - -__attribute__((constructor)) void triton_msg_ssm_init_register(void); - -__attribute__((constructor)) void triton_msg_ssm_init_register(void) -{ - triton_init_register("triton.net.ssm", triton_msg_ssm_init, triton_msg_ssm_finalize, NULL, "triton.net.message", "triton.resource.ssm"); -} - - -//TEST -struct triton_msg_method* get_msg_method() -{ - return &triton_msg_ssm_method; -} diff --git a/code/src/net/ssm/ssm-method.h b/code/src/net/ssm/ssm-method.h deleted file mode 100644 index 031e4e5..0000000 --- a/code/src/net/ssm/ssm-method.h +++ /dev/null @@ -1,8 +0,0 @@ -#ifndef __SSM_METHOD_H__ -#define __SSM_METHOD_H__ -#include "src/common/triton-error.h" - -triton_ret_t triton_msg_ssm_init(void); -void triton_msg_ssm_finalize(void); - -#endif //__SSM_METHOD_H__ diff --git a/code/src/net/ssm/ssm-tcp-transport.c b/code/src/net/ssm/ssm-tcp-transport.c deleted file mode 100644 index ae0b8a3..0000000 --- a/code/src/net/ssm/ssm-tcp-transport.c +++ /dev/null @@ -1,161 +0,0 @@ -#include "ssm-transport.h" -#include "src/remote/byteswap.h" -#include <string.h> - -#define JZ_DEBUG -static __attribute__((noinline)) void __jz_wait_for_debugger(const char* file, const char* function, int line, int rank, int once) -{ - static int count = 0; - - if(count && once) - return; - count++; - printf("process ( pid = %d | rank = %d ) is waiting in %s at %s:%d for debugger\n", - getpid(), rank, function, file, line); - fflush(stdout); - for(;;); -} - -#ifdef JZ_DEBUG - void __jz_wait_for_debugger(const char* file, const char* function, int line, int rank, int once); - #define JZ_WAIT_FOR_DEBUGGER() __jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 0) - #define JZ_WAIT_FOR_DEBUGGER_IF(cond) \ - do{if((cond))__jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 0);}while(0) - #define JZ_WAIT_ONCE_FOR_DEBUGGER() __jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 1) - #define JZ_WAIT_ONCE_FOR_DEBUGGER_IF(cond) \ - do{if((cond))__jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 1);}while(0) -#else - #define JZ_WAIT_FOR_DEBUGGER() - #define JZ_WAIT_FOR_DEBUGGER_IF() - #define JZ_WAIT_ONCE_FOR_DEBUGGER() - #define JZ_WAIT_ONCE_FOR_DEBUGGER_IF() -#endif -static uint64_t get_address(uint8_t* raw_address_data, void* additional_data) -{ - /* - The address is made of an IP address and a port - */ - uint64_t addr = ((uint64_t)raw_address_data[0]) << (40) | //IP - ((uint64_t)raw_address_data[1]) << (32)| //IP - ((uint64_t)raw_address_data[2]) << (24)| //IP - ((uint64_t)raw_address_data[3]) << (16); //IP - -#if __BYTE_ORDER == __LITTLE_ENDIAN - addr |= (((uint64_t)raw_address_data[5]) << (8) | //port - ((uint64_t)raw_address_data[4])); //port -#else - addr |= (((uint64_t)raw_address_data[4]) << (8) | //port - ((uint64_t)raw_address_data[5])); //port -#endif - - /*NOTE: The port is expected to be in the right node endianess in raw_address_data*/ - - (void)additional_data; - return addr; -} - -unsigned char *g_raw_data; -static uint64_t get_address_from_str_addr(char* str_addr, void* additional_data) -{ - uint8_t raw_address[6]; - (void)additional_data; - sscanf(str_addr, "tcp::%hhu.%hhu.%hhu.%hhu|%hu", &raw_address[0], &raw_address[1], &raw_address[2], - &raw_address[3], ((unsigned short*)(&raw_address[4]))); - g_raw_data = raw_address; - return get_address(raw_address, NULL); -} - -static void get_str_address(uint64_t address, char str_address[], void* additional_args) -{ - uint8_t ip[4]; - unsigned short port = address&0xffff; - address >>= 16; - ip[3]=(uint8_t)(address&0xff); - address >>= 8; - ip[2]=(uint8_t)(address&0xff); - address >>= 8; - ip[1]=(uint8_t)(address&0xff); - address >>= 8; - ip[0]=(uint8_t)(address&0xff); - - sprintf(str_address, "tcp::%d.%d.%d.%d|%d",(uint8_t)ip[0], (uint8_t)ip[1], (uint8_t)ip[2], - (uint8_t)ip[3], port); - (void)additional_args; -} - - -static inline int get_address_data_size() -{ - return 6; //6 bytes ... assuming IPv4 -} - -static ssm_Itp create_transport(void* args) -{ - return NULL; -} - -static inline int get_addr_serialization_size() -{ - return sizeof(uint8_t) + sizeof(uint8_t)*4 + sizeof(short); //assuming IPv4 -} - -static void get_serialized_address(ssm_Itp transport, uint8_t* address_data, uint8_t* serialization) -{ - //IP - serialization[0] = address_data[0]; - serialization[1] = address_data[1]; - serialization[2] = address_data[2]; - serialization[3] = address_data[3]; - - //port - *(int16_t*)(&(serialization[4])) = aehton16(*((short*)(&address_data[4]))); -} - - -static void get_raw_address_in_host_bytes(ssm_Itp transport, uint8_t* net_bytes_address, uint8_t* host_bytes) -{ - //IP - host_bytes[0] = net_bytes_address[0]; - host_bytes[1] = net_bytes_address[1]; - host_bytes[2] = net_bytes_address[2]; - host_bytes[3] = net_bytes_address[3]; - - //port - *(int16_t*)(&(host_bytes[4])) = aentoh16(*((short*)(&net_bytes_address[4]))); -} - -static void create_address_arg_data(uint8_t* raw_address_data, void* transport_address_arg_out) -{ - unsigned short port; - ssmptcp_addrargs_t* tcp_args = (ssmptcp_addrargs_t*)transport_address_arg_out; - tcp_args->host = (uint8_t*)transport_address_arg_out+sizeof(ssmptcp_addrargs_t); - sprintf(tcp_args->host, "%d.%d.%d.%d", (int)raw_address_data[0], (int)raw_address_data[1], - (int)raw_address_data[2], (int)raw_address_data[3]); - port = *((unsigned short*)(&raw_address_data[4])); - tcp_args->port = aentoh16(port); -} - -static ssm_Haddr create_addr_handle(ssm_Iaddr addr_interface, void* addr_args) -{ - ssmptcp_addrargs_t* tcp_args = (ssmptcp_addrargs_t*)addr_args; - return addr_interface->create(addr_interface, tcp_args); -} - -static ssm_transport_abstraction_t tcp_abstraction = -{ - .get_address = get_address, - .get_address_from_str_addr = get_address_from_str_addr, - .get_address_data_size = get_address_data_size, - .get_str_address = get_str_address, - .create_transport = create_transport, - .get_addr_serialization_size = get_addr_serialization_size, - .get_serialized_address = get_serialized_address, - .get_raw_address_in_host_bytes = get_raw_address_in_host_bytes, - .create_address_arg_data = create_address_arg_data, - .create_addr_handle = create_addr_handle -}; - -ssm_transport_abstraction_t* get_tcp_abstraction() -{ - return &tcp_abstraction; -} diff --git a/code/src/net/ssm/ssm-transport.h b/code/src/net/ssm/ssm-transport.h deleted file mode 100644 index fa48166..0000000 --- a/code/src/net/ssm/ssm-transport.h +++ /dev/null @@ -1,69 +0,0 @@ -#ifndef __SSM_TRANSPORT_H__ -#define __SSM_TRANSPORT_H__ - -/* -This file defines a transport specific interface for SSM use in triton -*/ - -#include "ssm.h" -#include "ssmptcp.h" - -#define SSM_MAX_ADDR_ARG_SIZE 256 - -typedef enum ssm_transport_type_t -{ - SSM_TCP, - SSM_UDP, - SSM_IB, - /* - etc. - */ -}ssm_transport_type_t; - -typedef struct ssm_transport_abstraction_t -{ - /* - get address from raw address data component laid out as a uint8_t[] - */ - uint64_t (*get_address)(uint8_t* address_data, void* additional_data); - - uint64_t (*get_address_from_str_addr)(char* str_addr, void* additional_data); - - /* - get the number of items in the uint8_t address_data[] representation of the address - */ - int (*get_address_data_size)(); - - /* - get the string representation of the address; i.e. the ssm://stuff_stuff form of the address - additional_args contains anything required by the transport to generate the - str_address from address - */ - void (*get_str_address)(uint64_t address, char* str_address, void* additional_args); - - ssm_Itp (*create_transport)(void*); //the prototype of this function will change for SURE. For now it will take a void* - - int (*get_addr_serialization_size)(); - - /*Just as the name says. - The serialization is in network byte order (See the TCP implementation for an example) - */ - void (*get_serialized_address)(ssm_Itp transport, uint8_t* address_data, uint8_t* serialization); - - /*Just as the name says. - The serialization is in network byte order (See the TCP implementation for an example) - */ - void (*get_raw_address_in_host_bytes)(ssm_Itp transport, uint8_t* net_bytes_address, - uint8_t* host_bytes); - - /*transport_address_arg_out must be a buffer of at least SSM_MAX_ADDR_ARG_SIZE - */ - void (*create_address_arg_data)(uint8_t* raw_address_data, void* transport_address_arg_out); - ssm_Haddr (*create_addr_handle)(ssm_Iaddr addr_interface, void* addr_args); - -}ssm_transport_abstraction_t; - - -ssm_transport_abstraction_t* get_tcp_abstraction(); - -#endif //__SSM_TRANSPORT_H__ diff --git a/code/src/net/tests/README_FOR_ONE_SIDED b/code/src/net/tests/README_FOR_ONE_SIDED deleted file mode 100644 index f6c5b7d..0000000 --- a/code/src/net/tests/README_FOR_ONE_SIDED +++ /dev/null @@ -1,65 +0,0 @@ - -I. General Description -II. Input - -I. GENERAL DESCRIPTION - *There is always at least 1 server -If there are more than 2 processes, all prime ranks are servers - *To ease the test output reading, files are used instead of stdout -At the end of the execution, each process generates a file named trace_{client/server}_rank_{rank}. -For instance, the process 4 will generate trace_client_rank_4 and the server 3 will generate -trace_server_rank_3. - *Each process has a reasonnably large I/O buffer that it initializes with its rank. -Then at the end of the execution, each server writes sections of its buffer where at least -a client did a write. A server that made no write will generate an empty trace file. -Clients generate trace content when they make reads. -The test makes each client read and write once at offsets that are defined in an input file. -(See the file sample_one_sided_test_input for an example of input and input directives) - *The chunk of trace below taken from trace_server_rank_1 shows/proves that a read operation -from the client of rank 9 succeeded. The read was non-contiguous and spans the iov {{5,5},{150,10}} -in a registered memory which started at offset 95000. As a reminder, iov_items are {offset, size}. -The ellipsis is inserted for brevity. -<-----TRACING MEMORY AFTER I/O WITH CLIENT 9-------> -95005:9 -95006:9 -95007:9 -95008:9 -95009:9 -95010:1 -95011:1 -95012:1 -95013:1 -95014:1 -... -... -95147:1 -95148:1 -95149:1 -95150:9 -95151:9 -95152:9 -95153:9 -95154:9 -95155:9 -95156:9 -95157:9 -95158:9 -95159:9 -</-----TRACING MEMORY AFTER I/O WITH CLIENT 9-------> - - *For each I/O, a client chooses the primary server randomly. -The pipelining strategy simply distributes iov items accross the server in a round-ronbin fashion. -The primary server does that job and always start from itself. -To ease the trace reading/checking of the pipelining, run a test with 4 processes max so as to have -a single client (1 non-primary-number rank which is 0). Then in the input file, choose small and easily -human-eye assessable ranges (e.g. 10s instead of hundreds of thousands of bytes). And make sure the ranges -for each I/O are disjoint. - The program should always output the rank of the -chosen primary server and allow you to understand the response map computation. For instance, with -4 processes, if rank 2 is selected as primary server, and the client has the 6 iov items -{{5,5},{12,8},{40,10},{100,10},{120,10},{150,10}} for a given I/O, -they will be distributed as follows {{5,5},{100,10}}, {{12,8},{120,10}} -{{40,10},{150,10}} respectively over the ranks 2,3 and 1. - -II. INPUT -see sample_one_sided_test_input diff --git a/code/src/net/tests/module.mk.in b/code/src/net/tests/module.mk.in deleted file mode 100644 index 27addcf..0000000 --- a/code/src/net/tests/module.mk.in +++ /dev/null @@ -1,18 +0,0 @@ -DIR := src/net/tests - -ifneq (,$(BUILD_MPI)) - -AETESTSRC += $(DIR)/mpi-client-server.ae\ - $(DIR)/mpi-one-sided.ae - -MODCFLAGS_$(DIR)/mpi-client-server = $(MPICFLAGS) -MODLDFLAGS_$(DIR)/mpi-client-server = $(MPILDFLAGS) -MODLIBS_$(DIR)/mpi-client-server = $(MPILIBS) -MODCC_$(DIR)/mpi-client-server = $(MPICC) - -endif # BUILD_MPI - -ifneq (,$(BUILD_SSM)) -AETESTSRC += $(DIR)/ssm-one-sided.ae\ - $(DIR)/ssm-sr-test.ae -endif # BUILD_SSM diff --git a/code/src/net/tests/mpi-client-server.ae b/code/src/net/tests/mpi-client-server.ae deleted file mode 100644 index d76f5df..0000000 --- a/code/src/net/tests/mpi-client-server.ae +++ /dev/null @@ -1,100 +0,0 @@ - -#include <mpi.h> - -#include "src/net/triton-message.hae" -#include "src/net/triton-message-method.hae" -#include "src/net/mpi/mpi-method.h" -#include "src/common/triton-debug.h" -#include "src/common/triton-init.h" - -#include <unistd.h> - -__blocking void do_server(void) -{ - triton_ret_t ret; - triton_msg_group_t unexp_group, exp_group; - char *buffer; - uint32_t buffer_size, recved; - triton_addr_t from, to, me; - triton_msg_tag_t tag; - int count = 0, total = 0; - triton_string_t nstr, mestr, tostr; - - ret = triton_msg_mpi_init(); - triton_error_assert(ret); - - triton_debug_enable("/dev/stdout", "all"); - ret = triton_msg_group_add("test-server-unexp", &unexp_group); - triton_error_assert(ret); - - ret = triton_msg_group_add("mpi", "test-server-exp", &exp_group); - triton_error_assert(ret); - - buffer_size = 16 * 1024; - buffer = malloc(buffer_size); - assert(buffer); - - me = triton_addr_self("mpi"); - triton_error_assert(ret); - ret = triton_addr_to_string(me, &mestr); - triton_error_assert(ret); - - MPI_Comm_size(MPI_COMM_WORLD, &total); - - if(!strcmp(mestr.string, "mpi://0")) - { - /* I'm a server! */ - while(count < (total-1)) - { - ret = triton_msg_recv_any(unexp_group, &from, &tag, 1, &buffer, &buffer_size, &recved); - triton_error_assert(ret); - ret = triton_addr_to_string(from, &nstr); - triton_error_assert(ret); - printf("Node %s received message from: %s, with tag: %d\n", mestr.string, nstr.string, tag); - ret = triton_msg_send(from, exp_group, tag, 1, &buffer, &recved); - printf("Node %s sent response to: %s, with tag: %d\n", mestr.string, nstr.string, tag); - - triton_string_destroy(&nstr); - triton_error_assert(ret); - count++; - } - } - else - { - ret = triton_addr_lookup("mpi://0", &to); - tag = triton_msg_new_tag(unexp_group); - - ret = triton_addr_to_string(to, &tostr); - triton_error_assert(ret); - - /* I'm a client */ - ret = triton_msg_send(to, unexp_group, tag, 1, &buffer, &buffer_size); - printf("Node %s sent message to: %s, with tag: %d\n", mestr.string, tostr.string, tag); - triton_error_assert(ret); - - printf("Node %s receiving message from: %s, with tag: %d\n", mestr.string, tostr.string, tag); - ret = triton_msg_recv(to, exp_group, tag, 1, &buffer, &buffer_size, &recved); - printf("Node %s received message from: %s, with tag: %d\n", mestr.string, tostr.string, tag); - - triton_string_destroy(&tostr); - triton_error_assert(ret); - triton_addr_free(to); - } - - triton_string_destroy(&mestr); - triton_addr_free(me); - triton_msg_mpi_finalize(); -} - -__blocking int aesop_main(int argc, char **argv) -{ - if(argc > 1) - { - sleep(30); - } - do_server(); - - return 0; -} - -aesop_main_set(aesop_main); diff --git a/code/src/net/tests/mpi-one-sided.ae b/code/src/net/tests/mpi-one-sided.ae deleted file mode 100644 index 55df79a..0000000 --- a/code/src/net/tests/mpi-one-sided.ae +++ /dev/null @@ -1,1024 +0,0 @@ -#include <aesop/aesop.h> -#include <mpi.h> -#include <stdio.h> -#include <errno.h> -#include "../triton-message-method.hae" -#include "../mpi/mpi-method.h" -#include <time.h> -#include <stdlib.h> -#include <string.h> -#include <stdarg.h> -#include <ctype.h> -#include <unistd.h> - - -#define MIN(a,b) ((a)<=(b)?(a):(b)) -#define MAX(a,b) ((a)>=(b)?(a):(b)) -#define ABORT_ON_MPI_FAILURE(ret,err_msg_format,...) \ -do \ -{ \ - if((ret) != MPI_SUCCESS) \ - { \ - triton_err(triton_log_default, "%s:%d: [MPI Proc rank %d]" err_msg_format, \ - __FILE__, __LINE__, rank, ##__VA_ARGS__); \ - MPI_Abort(MPI_COMM_WORLD, ret); \ - } \ -}while(0) - -#define KB 1024 -#define MB 1048576 - -#define PRINT(...) do{printf(__VA_ARGS__); fflush(stdout);}while(0) -#define TRACE(...) do{PRINT("%s:%d[%s]", __FILE__, __LINE__, __FUNCTION__); PRINT(__VA_ARGS__);}while(0) -#define TRACE_IF(cond,...) do{if((cond)) {TRACE(__VA_ARGS__);}}while(0) - -#define MAX_NB_SERVERS 256 /* For I'm using byte offsets and writing/reading char to check the test results. - I'm writing ranks to know who wrote where and who read from where; so - more than 255 is troublesome to visually assess the correctness of the test. - Writing char is just for the readability of the results of this test. - */ - - - - -#define MAX_DATA_SIZE 50*MB -int rank = -1; -int size = -1; -int nb_servers = -1; - -triton_addr_t self; -char str_self[51]; //to containg for instance "mpi://0" for rank 0 -triton_msg_group_t group; -triton_msg_tag_t tag; -triton_addr_t server_addrs[MAX_NB_SERVERS]; - -//TODO: Fix linked lists to use the triton ones - -typedef struct node_link_t -{ - struct node_link_t* next; -}node_link_t; - -static void add_node(void** node_head, void** node_tail, void* node) -{ - node_link_t **head = (node_link_t**)(node_head); - node_link_t **tail = (node_link_t**)(node_tail); - if(*tail) - { - (*tail)->next = (node_link_t*)node; - } - else - { - *head = (node_link_t*)node; - } - *tail = (node_link_t*)node; -} - -static void remove_node(void** node_head, void** node_tail, void* node) -{ - node_link_t *cur = (node_link_t*)(*node_head), *previous = NULL; - while(cur) - { - if(cur == node) - { - if(previous) - previous->next = cur->next; - else - *node_head = cur->next; - if(*node_tail == cur) - *node_tail = previous; - break; - } - previous = cur; - cur = cur->next; - } -} - -static int is_prime(int n) -{ - int i, half = n/2; - if(n == 0) - return 0; - if(n == 1) - return 1; - for(i=2; i<=half; i++) - if( (n%i) == 0) - return 0; - return 1; -} - -static inline int is_server(int process_rank) -{ - if(size <=2 ) - return process_rank >=size -1; - return is_prime(process_rank); -} - - -/*Just some arbitrary logic to decide who is client and who is server -*/ -static inline int is_local_process_a_server() -{ - return is_server(rank); -} - -static __blocking void open_endpoints() -{ - self = triton_addr_self("mpi"); - sprintf(str_self, "mpi://%d", rank); - triton_msg_group_add("mpi", "test-one-sided", &group); - tag = triton_msg_new_tag(group); -} - -static __blocking void close_endpoints() -{ - int i; - triton_addr_free(self); - for(i=0; i< nb_servers; i++) - triton_addr_free(server_addrs[i]); -} - -/* -Discovering servers could be implemented differently; e.g. having the -processes read from a file who's who -*/ -static __blocking void discover_servers(void) -{ - triton_ret_t tret; - int ret; - int i; - nb_servers = 0; - char str_addr[51]; - for(i=0; i< size; i++) - { - if(is_server(i)) - { - sprintf(str_addr, "mpi://%d", i); - tret = triton_addr_lookup(str_addr, &server_addrs[nb_servers++]); - triton_assert(tret == TRITON_SUCCESS); - } - } -} - - -FILE* trace_file = NULL; -#define TRACE_IN_FILE(...) do{fprintf(trace_file, __VA_ARGS__);}while(0) - -static void init_trace_in_file() -{ - char file_name[100]; - sprintf(file_name, "trace_%s_rank_%d", is_local_process_a_server()?"server":"client", - rank); - trace_file = fopen(file_name, "w"); -} - -static void close_trace_in_file() -{ - fclose(trace_file); -} - - -char* server_data = NULL; -char* read_server_data = NULL; -char* write_server_data = NULL; -triton_mem_reg_handle_t server_mrh; - -static void init_server_data() -{ - triton_ret_t tret; - if(!server_data) - { - server_data = triton_buffer_allocate(self, MAX_DATA_SIZE); - triton_assert(server_data); - tret = triton_register_memory(self, server_data, MAX_DATA_SIZE, &server_mrh); - triton_assert(tret == TRITON_SUCCESS); - } - memset(server_data, rank, MAX_DATA_SIZE); - write_server_data = read_server_data = server_data; - //write_server_data = &server_data[SERVER_DATA_SIZE/2]; -} - -static void free_server_data() -{ - triton_ret_t tret; - if(server_data) - { - tret = triton_unregister_memory(self, server_mrh); - triton_assert(tret == TRITON_SUCCESS); - triton_buffer_free(self, server_data); - server_data = NULL; - } -} - -/* -iov_items are assigned round robin starting from the primarily contacted server ... -*/ -static inline int get_nb_responding_servers(triton_iov_item_t* iov, int *iov_indices, int nb_iov_items) -{ - return MIN(nb_servers, nb_iov_items); -} - - -/* -iov_items are assigned round robin starting from the primarily contacted server ... -Returns the number of iov the server is expected to service -*/ -static int get_response_map_for_specific_server(int nb_iov_items, int* iov_indices, - triton_addr_t server_addr) -{ - /*start at address of primarily contacted server to ensure that it gets at least 1*/ - - int i, k; - int next_server_index = -1; - for(i=0; i<nb_servers; i++) - if(triton_addr_equal(server_addrs[i], self)) - { - next_server_index = i; - break; - } - triton_assert(next_server_index != -1); - for(i=0, k=0; i<nb_iov_items; i++) - { - if(triton_addr_equal(server_addrs[next_server_index], server_addr)) - iov_indices[k++] = i; - next_server_index = (next_server_index+1)%nb_servers; - } - return k; -} - -#define TEST_TYPE_READ 0 -#define TEST_TYPE_WRITE 1 -#define TEST_TYPE_ACKS 2 -#define TEST_TYPE_READ_SCHEDULE_TRANSFER_REQUEST 4 -#define TEST_TYPE_WRITE_SCHEDULE_TRANSFER_REQUEST 6 -#define TEST_TYPE_ACK_FRAGMENT 7 - -#define TEST_TYPE_SHUTDOWN 10 - -#define MAX_RPC_SIZE 4*KB -#define MAX_NB_IOV_ITEMS 50 - -typedef enum io_type_t -{ - IOT_UNDEFINED, - IOT_READ, - IOT_WRITE -}io_type_t; - - -static __blocking void notify_for_client_shutdown() -{ - int i; - triton_ret_t tret; - char payload_content = TEST_TYPE_SHUTDOWN; - char* payload = &payload_content; - uint32_t packet_size = sizeof(char); - for(i=0; i<nb_servers; i++) - { - tret = triton_msg_send(server_addrs[i], group, tag, 1, &payload, &packet_size); - triton_assert(tret == TRITON_SUCCESS); - } -} - -/*Scan all the iov_items and compute an offset and size that will exactly include -them all. -out_mem_pub_params contains the offset and size of the required mem_pub_handle -*/ -static void get_io_address_extent(triton_iov_item_t* iovs, int nb_iov_items, - triton_iov_item_t* out_mem_pub_params) -{ - size_t smallest_start = iovs[0].offset, temp_start, largest_end = iovs[0].offset + iovs[0].size, temp_end; - int i; - for(i=1; i<nb_iov_items; i++) - { - temp_start = iovs[i].offset; - temp_end = temp_start + iovs[i].size; - smallest_start = MIN(smallest_start, temp_start); - largest_end = MAX(largest_end, temp_end); - } - out_mem_pub_params->offset = smallest_start; - out_mem_pub_params->size = largest_end - smallest_start; -} - -#if 0 -int *g_rpc_size; -char *g_rpc_packet; -triton_iov_item_t* g_iovs; -int g_nb_iov_items; -char* g_serialized_mem_pub; -char* g_str_client_addr; -uint64_t g_unique_token; -uint32_t *g_offset; -int g_serialized_mem_pub_size; -#endif - -/* -unique_token is used to identify a specific request. Two pairs (str_client_addr, unique_token) -will never exist at the same time. -*/ -static int build_io_rpc(triton_iov_item_t* iovs, int* valid_iov_indices, int nb_iov_items, - char* serialized_mem_pub, int serialized_mem_pub_size, char* str_client_addr, uint64_t unique_token, - char* rpc_packet) -{ - triton_iov_item_t partial_iov[MAX_NB_IOV_ITEMS]; - int i; - for(i=0; i<nb_iov_items; i++) - { - partial_iov[i].offset = iovs[valid_iov_indices[i]].offset; - partial_iov[i].size = iovs[valid_iov_indices[i]].size; - } - - int rpc_size = 0; - - strcpy(rpc_packet, str_client_addr); - rpc_size += (strlen(str_client_addr)+1); - - memcpy(rpc_packet + rpc_size, &unique_token, sizeof(uint64_t)); - rpc_size+=sizeof(uint64_t); - - memcpy(rpc_packet + rpc_size, &serialized_mem_pub_size, sizeof(int)); - rpc_size+=sizeof(int); - - memcpy(rpc_packet + rpc_size, serialized_mem_pub, serialized_mem_pub_size); - rpc_size += serialized_mem_pub_size; - - memcpy(rpc_packet + rpc_size, &nb_iov_items, sizeof(nb_iov_items)); - rpc_size += sizeof(nb_iov_items); - - memcpy(rpc_packet + rpc_size, partial_iov, sizeof(triton_iov_item_t)*nb_iov_items); - rpc_size += sizeof(triton_iov_item_t)*nb_iov_items; - - return rpc_size; -} - - -static void extract_io_rpc_data(char* rpc_packet, triton_iov_item_t** iovs, int *nb_iov_items, - int *serialized_mem_pub_size, char** serialized_mem_pub, char **str_client_addr, uint64_t *unique_token) -{ - uint32_t offset = 0; - - *str_client_addr = rpc_packet; - offset += (strlen(*str_client_addr) + 1); - - *unique_token = *(uint64_t*)(rpc_packet + offset); - offset += sizeof(uint64_t); - - - *serialized_mem_pub_size = *((int*)(rpc_packet + offset)); - offset += sizeof(int); - - *serialized_mem_pub = (char*)(rpc_packet + offset); - offset += *serialized_mem_pub_size; - - *nb_iov_items = *((int*)(rpc_packet + offset)); - offset += sizeof(int); - - *iovs = (triton_iov_item_t*)(rpc_packet + offset); -} - - -static __blocking void do_client_io(char* mem_start, triton_mem_reg_handle_t mrh, triton_iov_item_t* iovs, - int nb_iov_items, io_type_t iot) -{ - triton_iov_item_t mem_pub_handle_params; - triton_addr_t primary_server_addr = server_addrs[rand()%nb_servers]; - int ret; - int i; - - triton_ret_t tret; - triton_mem_pub_handle_t mph; - MPI_Status status; - char some_buf[MAX_RPC_SIZE]; - char rpc_packet[MAX_RPC_SIZE]; //we're not really doing RPC here but this packet will contain approximately - //the same data as the equivalent RPC payload - char* packet_ptr = NULL; //just to make gcc happy; see where it is used - int serialized_mem_pub_h_size; - uint32_t rpc_size = 0; - uint32_t byte_received = 0; - - get_io_address_extent(iovs, nb_iov_items, &mem_pub_handle_params); - - int *valid_iov_indices = malloc(nb_iov_items*sizeof(int)); - for(i=0; i<nb_iov_items; i++) - valid_iov_indices[i] = i; - - tret = triton_publish_memory(self, mrh, mem_pub_handle_params.offset, mem_pub_handle_params.size, &mph); - triton_assert(tret == TRITON_SUCCESS); - - serialized_mem_pub_h_size = triton_get_serialized_handle_size(self, MEM_PUB_HANDLE_TYPE, &mph); - tret = triton_get_serialized_handle(self, MEM_PUB_HANDLE_TYPE, (void*)&mph, some_buf); - triton_assert(tret == TRITON_SUCCESS); - - rpc_size = build_io_rpc(iovs, valid_iov_indices, nb_iov_items, some_buf, serialized_mem_pub_h_size, - str_self, (uint64_t)mph, &rpc_packet[1]); - free(valid_iov_indices); - rpc_packet[0] = iot == IOT_READ ? TEST_TYPE_READ : TEST_TYPE_WRITE; - rpc_size++; - - //issue request - packet_ptr = rpc_packet; - PRINT("[CLIENT %s] about to issue %s request %lu\n", str_self, iot == IOT_READ ?"READ":"WRITE", - (uint64_t)mph); - tret = triton_msg_send(primary_server_addr, group, tag, 1, &packet_ptr, &rpc_size); - triton_assert(tret == TRITON_SUCCESS); - - //receive ack - tret = triton_msg_recv(primary_server_addr, group, tag, 1, &packet_ptr, &rpc_size, &byte_received); - triton_assert(tret == TRITON_SUCCESS); - triton_assert(rpc_packet[0] == TEST_TYPE_ACKS); - - tret = triton_wait(self, mph, (void*)(rpc_packet+sizeof(char))); - triton_assert(tret == TRITON_SUCCESS); - - tret = triton_unpublish_memory(self, mph); - triton_assert(tret == TRITON_SUCCESS); - - if(iot == IOT_READ) - { - //Display the whole region covered by the mem_pub_handle - PRINT("[CLIENT %s] DONE READING STUFF \n", str_self); - for(i=mem_pub_handle_params.offset; i<mem_pub_handle_params.size+mem_pub_handle_params.offset; i++) - TRACE_IN_FILE("%d: %d\n", i, mem_start[i]); - TRACE_IN_FILE("\n\n"); - } - else - { - TRACE_IN_FILE("[CLIENT %s] DONE WRITING STUFF ", str_self); - PRINT("[CLIENT %s] DONE WRITING STUFF \n", str_self); - } - - TRACE_IN_FILE("\n"); -} - - -FILE* input_file = NULL; -char input_file_name[201]; - -static void open_input() -{ - input_file = fopen(input_file_name, "r"); - triton_assert(input_file); -} - -static void close_input() -{ - fclose(input_file); -} - -/* -Returns 1 if this process has iov. 0 otherwise. -If this process has iovs, then the file read pointer is set -to the begining of its iovs -*/ -static int has_iov() -{ - char buf[1025]; - char str_rank[5]; - int addr_length; - sprintf(str_rank, "%d", rank); - addr_length = strlen(str_rank); - while(fgets(buf, 1024, input_file)) - { - if((strlen(buf) > addr_length + 2) && buf[0] == '<' && strncmp(buf+1, str_rank, addr_length) == 0) - return 1; - } - return 0; -} - - -static void extract_iov_item(char* buf, triton_iov_item_t* iov_item) -{ - int i; - char *token = NULL; - int j=0; - for(j=0; j<2; j++) - { - token = strtok(j?NULL:buf, ","); - while(!isdigit(*token)) - token++; - i = strlen(token); - i--; - while(!isdigit(token[i])) - i--; - token[i+1] = '\0'; - if(j == 0) - iov_item->offset = (uint64_t)atol(token); - else - iov_item->size = (uint64_t)atol(token); - } -} - -typedef enum input_extract_phase_t -{ - UNDEFINED, - NB_IOV_ITEMS, - IOV_ITEMS -}input_extract_phase_t; - -/* -Extract a single iov for a single operation (READ or WRITE) -If iot == IOT_UNDEFINED at the exit of the function, then no more to read. -*/ -static int extract_iov(triton_iov_item_t** iov, io_type_t* iot) -{ - char buf[1024]; - char *line = buf; - char end_of_iov[51]; - int nb_iov_items = 0; - input_extract_phase_t input_extract_phase = UNDEFINED; - int nb_extracted_iov_items = 0; - - sprintf(end_of_iov, "</%d>", rank); - *iot = IOT_UNDEFINED; - while(fgets(line, 1024, input_file)) - { - while(*line == ' ' || *line == '\t') - line++; - if(*line == '\n' || *line == '#') - continue; - if(strncmp(line, end_of_iov, strlen(end_of_iov)) == 0) - return nb_extracted_iov_items; - if(strncmp(line, "<READ>", strlen("<READ>")) == 0) - { - *iot = IOT_READ; - input_extract_phase = NB_IOV_ITEMS; - continue; - } - else if(strncmp(line, "<WRITE>", strlen("<WRITE>")) == 0) - { - *iot = IOT_WRITE; - input_extract_phase = NB_IOV_ITEMS; - continue; - } - else if(strncmp(line, "</READ>", strlen("</READ>")) == 0 || - strncmp(line, "</WRITE>", strlen("</WRITE>")) == 0) - return nb_extracted_iov_items; - if(isdigit(line[0])) - { - buf[strlen(buf)-1]='\0'; - if(input_extract_phase == NB_IOV_ITEMS) - { - nb_iov_items = atoi(buf); - *iov = malloc(nb_iov_items * sizeof(triton_iov_item_t)); - triton_assert(*iov); - input_extract_phase = IOV_ITEMS; - } - else if(input_extract_phase == IOV_ITEMS) - { - extract_iov_item(buf, &((*iov)[nb_extracted_iov_items++])); - } - } - } - return nb_extracted_iov_items; -} - - -/* -*/ -static __blocking void enter_client_loop() -{ - char *buffer = NULL; - triton_ret_t tret; - triton_mem_reg_handle_t mrh1; - io_type_t iot; - int nb_iov_items; - - buffer = malloc(MAX_DATA_SIZE*sizeof(char)); - triton_assert(buffer); - - triton_iov_item_t *iov = NULL; - - memset(buffer, rank, MAX_DATA_SIZE); //initialize the client memory - - tret = triton_register_memory(self, buffer, MB, &mrh1); - triton_assert(tret == TRITON_SUCCESS); - - pwait - { - pprivate triton_iov_item_t* iov_for_a_pbranch; - if(has_iov()) - { - /*There is a bit of a bad practice here. The iov is allocated in extract_iov - and freed in the pbranch. - */ - nb_iov_items = extract_iov(&iov, &iot); - while( nb_iov_items ) - { - iov_for_a_pbranch = iov; //this because the pointer will be lost in the - //next extract_iov call - pbranch - { - do_client_io(buffer, mrh1, iov_for_a_pbranch, nb_iov_items, iot); - free(iov_for_a_pbranch); - } - nb_iov_items = extract_iov(&iov, &iot); - } - } - } - - tret = triton_unregister_memory(self, mrh1); - triton_assert(tret == TRITON_SUCCESS); - - notify_for_client_shutdown(); - free(buffer); -} - -static void display_server_side_read_data(char* buf_start, triton_iov_item_t *iov, int nb_iov_items, - char* str_client_addr) -{ - int i; - triton_iov_item_t data_span; - get_io_address_extent(iov, nb_iov_items, &data_span); - TRACE_IN_FILE("<-----TRACING MEMORY AFTER I/O WITH CLIENT %s------->\n", str_client_addr); - for(i=data_span.offset; i<data_span.offset+data_span.size; i++) - { - TRACE_IN_FILE("%d:%d\n", i, buf_start[i]); - } - TRACE_IN_FILE("</-----TRACING MEMORY AFTER I/O WITH CLIENT %s------->\n\n", str_client_addr); -} - -static __blocking void do_server_io(triton_addr_t client_addr, triton_mem_pub_handle_t mem_pub_handle, - triton_iov_item_t *iov, int nb_iov_items, int8_t test_type) -{ - int i; - triton_ret_t tret; - - char* buf_start = test_type == TEST_TYPE_READ? read_server_data: write_server_data; - char **bufs = malloc(sizeof(char*)*nb_iov_items); - triton_assert(bufs); - for(i=0; i<nb_iov_items; i++) - bufs[i] = (char*)(buf_start + iov[i].offset); - - if(test_type == TEST_TYPE_READ) - { - tret = triton_msg_put(self, client_addr, mem_pub_handle,iov, bufs, nb_iov_items); - } - else - { - tret = triton_msg_get(self, client_addr, mem_pub_handle, iov, bufs, nb_iov_items); - } - triton_assert(tret == TRITON_SUCCESS); - free(bufs); -} - -/* -Wait infos are kept by the primarily contacted server. -Each secondarily contacted server sends its ack to the -primarily contacted server with a unique token that identifies the -request the ack is being sent for. -nb_expected_acks_from_other_servers is decremented by the primary server upon each reception -of unique_token until it reaches 0. Then the primarily contacted server sends the final ack to the -client and releases the wait info object -*/ -typedef struct server_side_wait_info_t -{ - struct server_side_wait_info_t *next; - triton_addr_t client_addr; - int nb_expected_acks_from_other_servers; - uint64_t unique_token; -}server_side_wait_info_t; - - -server_side_wait_info_t* wait_info_head = NULL; -server_side_wait_info_t* wait_info_tail = NULL; - - -static void add_wait_info(triton_addr_t client_addr, int nb_expected_acks_from_other_servers, uint64_t unique_token) -{ - server_side_wait_info_t* wait_info = malloc(sizeof(server_side_wait_info_t)); - triton_assert(wait_info); - wait_info->client_addr = client_addr; - wait_info->next = NULL; - wait_info->nb_expected_acks_from_other_servers = nb_expected_acks_from_other_servers; - wait_info->unique_token = unique_token; - add_node((void**)(&wait_info_head), (void**)(&wait_info_tail), (void*)wait_info); -} - -static server_side_wait_info_t* retrieve_wait_info(uint64_t unique_token) -{ - server_side_wait_info_t* wait_info = wait_info_head; - while(wait_info) - { - if(wait_info->unique_token == unique_token) - return wait_info; - wait_info = wait_info->next; - } - return NULL; -} - -static inline void remove_wait_info(server_side_wait_info_t* wait_info) -{ - remove_node((void**)(&wait_info_head), (void**)(&wait_info_tail), (void*)wait_info); - free(wait_info); -} - -static __blocking void enter_server_loop() -{ - int nb_client_shutdowns = 0; - volatile int clear_to_read_next_incoming_rpc = 1; - volatile int request_is_received; - triton_mutex_t mutex; - triton_mutex_t rpc_mutex; - char listener_rpc_packet[MAX_RPC_SIZE]; - char* listener_packet_ptr = listener_rpc_packet; - - triton_mutex_init(&mutex, NULL); //to protect the shutdown variable and possibly other stuff - triton_mutex_init(&rpc_mutex, NULL); //to protect the shutdown variable and possibly other stuff - - pwait - { - pprivate int serialized_mem_pub_h_size; - pprivate char rpc_packet[MAX_RPC_SIZE]; - char* packet_ptr = rpc_packet; //just to make gcc happy - pprivate triton_iov_item_t* iovs; - pprivate char* serialized_mem_pub; - pprivate triton_addr_t client_addr; - pprivate char* str_client_addr; - pprivate uint32_t rpc_size; - pprivate uint64_t unique_token; //used to differentiate requests from the same client. - - pprivate int iov_indices[MAX_NB_IOV_ITEMS]; - pprivate char io_rpc_packets[MAX_NB_SERVERS][MAX_RPC_SIZE]; - pprivate int ret; - pprivate triton_ret_t tret; - pprivate triton_mem_pub_handle_t mem_pub_handle; - pprivate int nb_scheduled_servers = 0; - pprivate int i; - pprivate int j; - pprivate int nb_iov_items; - pprivate int nb_iov_items_for_specific_server; - pprivate server_side_wait_info_t* wait_info; - triton_iov_item_t partial_iov[MAX_NB_IOV_ITEMS]; - - triton_addr_t requester_addr; - - pprivate triton_addr_t private_requester_addr; - pprivate triton_msg_tag_t unused_tag; - pprivate uint32_t count; - pprivate uint32_t max_receive_size = MAX_RPC_SIZE; - pprivate uint32_t received_bytes; - - for(;;) - { - if(nb_client_shutdowns == size - nb_servers) - { - PRINT("Server %s is shutting down. Hasta luego!\n", str_self); - break; - } - - /* - The server waits for the last spawned pbranch to copy the rpc data before it goes - on listening to the next one again. The wait is necessary since the same - buffer is used. - */ - while(!clear_to_read_next_incoming_rpc); /*might use sleep or a semaphore, etc. instead here ... - ... thinking that the wait will usually be short. - (see below at the begining of the pbracnh)*/ - - clear_to_read_next_incoming_rpc = 0; - request_is_received = 0; - pbranch - { - tret = triton_msg_recv_any(group, &requester_addr, &unused_tag, 1, - &listener_packet_ptr, &max_receive_size, &received_bytes); - triton_assert(tret == TRITON_SUCCESS); - request_is_received = 1; - } - while(!request_is_received) - usleep(1000); - request_is_received = 0; - - pbranch - { - memcpy(rpc_packet, listener_rpc_packet, MAX_RPC_SIZE); - memcpy(&private_requester_addr, &requester_addr, sizeof(triton_addr_t)); - clear_to_read_next_incoming_rpc = 1; - - switch(rpc_packet[0]) - { - case TEST_TYPE_READ: - case TEST_TYPE_WRITE: - extract_io_rpc_data(&rpc_packet[1], &iovs, &nb_iov_items, &serialized_mem_pub_h_size, &serialized_mem_pub, - &str_client_addr, &unique_token); - PRINT("Primary Server %s handling I/O %lu\n", str_self, unique_token); - client_addr = private_requester_addr; - nb_scheduled_servers = get_nb_responding_servers(iovs, iov_indices, nb_iov_items); - if(nb_scheduled_servers > 1) - add_wait_info(client_addr, nb_scheduled_servers - 1, unique_token); - - /* - We adopt the following simple response map strategy. Starting from itself, - this primarily contacted server assigns each iov_item to the next server address (cycling) - until all the iov_items are assigned. - */ - for(i=0, j=0; nb_scheduled_servers > 1 && i<nb_servers; i++) - { - char* io_packet_ptr; - if(triton_addr_equal(server_addrs[i], self)) - continue; //handle own stuff in last position - nb_iov_items_for_specific_server = get_response_map_for_specific_server( - nb_iov_items, iov_indices, server_addrs[i]); - if(nb_iov_items_for_specific_server == 0) - continue; //the response map does not concern server_addrs[i] - rpc_size = build_io_rpc(iovs, iov_indices, nb_iov_items_for_specific_server, - serialized_mem_pub, serialized_mem_pub_h_size, str_client_addr, - unique_token, &io_rpc_packets[j][1]); - io_rpc_packets[j][0] = rpc_packet[0] == TEST_TYPE_READ? - TEST_TYPE_READ_SCHEDULE_TRANSFER_REQUEST: - TEST_TYPE_WRITE_SCHEDULE_TRANSFER_REQUEST; - rpc_size++; - io_packet_ptr = io_rpc_packets[j]; - tret = triton_msg_send(server_addrs[i], group, tag, 1, &io_packet_ptr, &rpc_size); - triton_assert(tret == TRITON_SUCCESS); - j++; - } - - /*This server sends its share of responses now*/ - tret = triton_create_handle_from_serialization(self, MEM_PUB_HANDLE_TYPE, (void*)(&mem_pub_handle), - serialized_mem_pub); - triton_assert(tret == TRITON_SUCCESS); - nb_iov_items_for_specific_server = get_response_map_for_specific_server( - nb_iov_items, iov_indices, self); - for(i=0; i<nb_iov_items_for_specific_server; i++) - { - partial_iov[i].offset = iovs[iov_indices[i]].offset; - partial_iov[i].size = iovs[iov_indices[i]].size; - } - PRINT("Primary Server %s about to serve I/O %lu\n", - str_self, unique_token); - tret = triton_start_epoch(self, mem_pub_handle); - triton_assert(tret == TRITON_SUCCESS); - do_server_io(client_addr, mem_pub_handle, partial_iov, - nb_iov_items_for_specific_server, rpc_packet[0] /*READ or WRITE*/); - tret = triton_end_epoch(self, mem_pub_handle); - triton_assert(tret == TRITON_SUCCESS); - PRINT("Primary Server %s done serving I/O %lu\n", - str_self, unique_token); - if(rpc_packet[0] == TEST_TYPE_WRITE) - { - display_server_side_read_data(server_data, partial_iov, - nb_iov_items_for_specific_server, str_client_addr); - } - tret = triton_free_serialized_handle(self, MEM_PUB_HANDLE_TYPE, - (void*)(&mem_pub_handle)); - triton_assert(tret == TRITON_SUCCESS); - if(nb_scheduled_servers == 1) - { - //send ack to client right away - rpc_packet[0]=TEST_TYPE_ACKS; - memcpy(&rpc_packet[1], &unique_token, sizeof(unique_token)); - rpc_size = 1 + sizeof(unique_token); - PRINT("Primary Server %s about to acknowledge I/O %lu\n", - str_self, unique_token); - packet_ptr = rpc_packet; - tret = triton_msg_send(client_addr, group, tag, 1, - &packet_ptr, &rpc_size); - triton_assert(tret == TRITON_SUCCESS); - PRINT("Primary Server %s completed and acknowledged I/O %lu\n", - str_self, unique_token); - } - break; - - //Handle pipeline response requests from some primarily contacted server - case TEST_TYPE_READ_SCHEDULE_TRANSFER_REQUEST: - case TEST_TYPE_WRITE_SCHEDULE_TRANSFER_REQUEST: - extract_io_rpc_data(&rpc_packet[1], &iovs, &nb_iov_items, &serialized_mem_pub_h_size, &serialized_mem_pub, - &str_client_addr, &unique_token); - PRINT("Auxiliary Server %s received pipelined request for I/O %lu\n", - str_self, unique_token); - - tret = triton_create_handle_from_serialization(self, MEM_PUB_HANDLE_TYPE, (void*)(&mem_pub_handle), - serialized_mem_pub); - triton_assert(tret == TRITON_SUCCESS); - tret = triton_addr_lookup(str_client_addr, &client_addr); - triton_assert(tret); - tret = triton_start_epoch(self, mem_pub_handle); - triton_assert(tret == TRITON_SUCCESS); - do_server_io(client_addr, mem_pub_handle, iovs, nb_iov_items, - rpc_packet[0] == TEST_TYPE_READ_SCHEDULE_TRANSFER_REQUEST? - TEST_TYPE_READ:TEST_TYPE_WRITE); - tret = triton_end_epoch(self, mem_pub_handle); - triton_assert(tret == TRITON_SUCCESS); - if(rpc_packet[0] == TEST_TYPE_WRITE_SCHEDULE_TRANSFER_REQUEST) - { - display_server_side_read_data(server_data, iovs, nb_iov_items, str_client_addr); - } - tret = triton_free_serialized_handle(self, MEM_PUB_HANDLE_TYPE, - (void*)(&mem_pub_handle)); - triton_assert(tret == TRITON_SUCCESS); - rpc_packet[0]=TEST_TYPE_ACK_FRAGMENT; - memcpy(&rpc_packet[1], &unique_token, sizeof(unique_token)); - rpc_size = 1 + sizeof(unique_token); - packet_ptr = rpc_packet; - tret = triton_msg_send(private_requester_addr/*primary server in this case*/, group, - tag, 1, &packet_ptr, &rpc_size); - triton_assert(tret == TRITON_SUCCESS); - triton_addr_free(client_addr); - break; - - /*Below is primarily contacted server receiving "ack fragments" from other - contacted servers for pipelining*/ - case TEST_TYPE_ACK_FRAGMENT: - unique_token = *((uint64_t*)(&rpc_packet[1])); - triton_mutex_lock(&mutex); - wait_info = retrieve_wait_info(unique_token); - triton_mutex_unlock(&mutex); - triton_assert(wait_info); - if(--wait_info->nb_expected_acks_from_other_servers == 0) - { - rpc_packet[0]=TEST_TYPE_ACKS; - memcpy(&rpc_packet[1], &unique_token, sizeof(unique_token)); - rpc_size = 1 + sizeof(unique_token); - packet_ptr = rpc_packet; - tret = triton_msg_send(wait_info->client_addr, group, tag, 1, &packet_ptr, &rpc_size); - triton_assert(tret == TRITON_SUCCESS); - PRINT("Primary Server %s acknowledging I/O %lu after pipelined transfers\n", - str_self, unique_token); - - triton_mutex_lock(&mutex); - remove_wait_info(wait_info); - triton_mutex_unlock(&mutex); - } - break; - - case TEST_TYPE_SHUTDOWN: - triton_mutex_lock(&mutex); - nb_client_shutdowns++; - PRINT("Server %s received client shutdown request no %d\n", str_self, - nb_client_shutdowns); - triton_mutex_unlock(&mutex); - break; - default: - PRINT("Server %s What the heck ...\n", str_self); - triton_assert(0); - } - } //pbranch - } //for - } //end pwait - triton_mutex_destroy(&mutex); - triton_mutex_destroy(&rpc_mutex); -} - -static void print_test_description() -{ - if(rank != 0) - return; - printf("\n\nThe program requires a unique string argument which is the input file name\n" - "See README_FOR_ONE_SIDED in \x1b[34mtriton/code/src/net/tests/\x1b[0m.\n" - "A sample input file named sample_one_sided_test_input can be copied from the aforementioned location as well\n\n"); -} - -__blocking int entry_point(int argc, char** argv) -{ - int dummy; - srand(time(NULL)); - MPI_Comm_rank(MPI_COMM_WORLD, &rank); - MPI_Comm_size(MPI_COMM_WORLD, &size); - if(argc != 2) - { - if(rank == 0) - { - print_test_description(); - } - return 0; - } - if(size > 256) - { - if(rank == 0) - { - printf("This test modifies memory at byte granularity. In order to allow the complete visualization of\n" - "which process modified which other process' data, the written data are process ranks. We needed\n" - "to fit all the ranks in a byte. The maximum allowed job size is therefore 256. The net-module \n" - "can handle virtually any job size. This limit of 256 concerns only the test and is for easiness of \n" - "interpretation only"); - } - return 0; - } - strcpy(input_file_name, argv[1]); - open_endpoints(); - open_input(); - - if(size == 1) - { - printf("\nHello I'm a jobless server ... who's going to bug me if there is no client, hein?\n"); - return 0; - } - discover_servers(); - init_trace_in_file(); - if(!is_local_process_a_server()) - { - enter_client_loop(); - } - else - { - init_server_data(); - enter_server_loop(); - free_server_data(); - } - close_trace_in_file(); - close_input(); - close_endpoints(); - return 0; -} - -aesop_main_set(entry_point); diff --git a/code/src/net/tests/mpi-server.ae b/code/src/net/tests/mpi-server.ae deleted file mode 100644 index f562cb7..0000000 --- a/code/src/net/tests/mpi-server.ae +++ /dev/null @@ -1,45 +0,0 @@ - -#include "src/net/triton-message.hae" - -__blocking void do_server(int buffer_size) -{ - triton_ret_t ret; - triton_msg_ctx_t mpi_ctx; - triton_msg_group_t unexp_group; - char *buffer; - - triton_msg_init(); - - ret = triton_msg_group_add(mpi_ctx, "test-server-unexp", &unexp_group); - triton_error_assert(ret); - - ret = triton_msg_group_add(mpi_ctx, "test-server-exp", &exp_group); - - buffer = malloc(buffer_size); - assert(buffer); - - ret = triton_msg_recv(mpi_ctx, TRITON_NODE_ANY, unexp_group, TRITON_MSG_ANY_TAG, 1, buffer, &buffer_size); - triton_error_assert(ret); - - -static int done = 0; -static void cb(void *ptr) -{ - done = 1; - ae_poll_break(NULL); -} - -int main(int argc, char *argv[]) -{ - ae_op_id_t op_id; - int bufsize = atoi(argv[1]); - - ret = do_server_post(bufsize, cp, NULL, NULL, NULL, &op_id); - - while(!done) - { - ae_poll(NULL, 0); - } - - return 0; -} diff --git a/code/src/net/tests/sample_one_sided_test_input b/code/src/net/tests/sample_one_sided_test_input deleted file mode 100644 index a2eef1e..0000000 --- a/code/src/net/tests/sample_one_sided_test_input +++ /dev/null @@ -1,53 +0,0 @@ -#Any line STARTING with '#' is ignored. '#' is to be used only at the starting of lines; no inline '#' is supported -# -#The file must respect the following restrictions: -# Avoid very long lines of comment. Typically, avoid lignes of more than 1000 characters -# A single tag on each line -# Tags must be closed just like in a markup language and NO standalone tag -# ( <TAG/>) even if the content must be empty. E.g. no <READ/> to mean -# no read iov. -# No space between ankle bracket and the tag identifier. E.g. <READ > or < READ> are not supported -# A single iov item on each line -# Empty lines are allowed; they are just ignored -# - -<0> -<READ> -#The bnumber right below is the number of iov items in this read -8 -5,5 -10,12 -40,10 -100,10 -120,10 -150,10 -512,128 -768,20 -</READ> -<READ> -1 -130,5 -</READ> -<WRITE> -3 -110,2 -200,1 -1000,10 -</WRITE> -<0> - -<4> -<READ> -2 -2048,1024 -4096, 10 -</READ> -<WRITE> -5 -512,8 -600,10 -700,100 -850,1 -900,1 -</WRITE> -<4> diff --git a/code/src/net/tests/ssm-one-sided.ae b/code/src/net/tests/ssm-one-sided.ae deleted file mode 100644 index 635b1f9..0000000 --- a/code/src/net/tests/ssm-one-sided.ae +++ /dev/null @@ -1,1161 +0,0 @@ -#include "src/net/ssm/ssm-method.h" - -#include <aesop/aesop.h> -#include <mpi.h> -#include <stdio.h> -#include <errno.h> -#include "../triton-message-method.hae" -#include <time.h> -#include <stdlib.h> -#include <string.h> -#include <stdarg.h> -#include <ctype.h> -#include <unistd.h> -#include <stdlib.h> - - -#define JZ_DEBUG -static __attribute__((noinline)) void __jz_wait_for_debugger(const char* file, const char* function, int line, int rank, int once) -{ - static int count = 0; - - if(count && once) - return; - count++; - printf("process ( pid = %d | rank = %d ) is waiting in %s at %s:%d for debugger\n", - getpid(), rank, function, file, line); - fflush(stdout); - for(;;); -} - -#ifdef JZ_DEBUG - #define JZ_WAIT_FOR_DEBUGGER() __jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 0) - #define JZ_WAIT_FOR_DEBUGGER_IF(cond) \ - do{if((cond))__jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 0);}while(0) - #define JZ_WAIT_ONCE_FOR_DEBUGGER() __jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 1) - #define JZ_WAIT_ONCE_FOR_DEBUGGER_IF(cond) \ - do{if((cond))__jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 1);}while(0) -#else - #define JZ_WAIT_FOR_DEBUGGER() - #define JZ_WAIT_FOR_DEBUGGER_IF() - #define JZ_WAIT_ONCE_FOR_DEBUGGER() - #define JZ_WAIT_ONCE_FOR_DEBUGGER_IF() -#endif - -#define ENV_ETHERNET_IP "ETHERNET_IP" -#define ENV_ETHERNET_PORT "ETHERNET_PORT" -#define ENV_PORT_BASE "PORT_BASE" -#define ETHERNET_LOOPBACK "127.0.0.1" - -#define MIN(a,b) ((a)<=(b)?(a):(b)) -#define MAX(a,b) ((a)>=(b)?(a):(b)) -#define ABORT_ON_MPI_FAILURE(ret,err_msg_format,...) \ -do \ -{ \ - if((ret) != MPI_SUCCESS) \ - { \ - triton_err(triton_log_default, "%s:%d: [MPI Proc rank %d]" err_msg_format, \ - __FILE__, __LINE__, rank, ##__VA_ARGS__); \ - MPI_Abort(MPI_COMM_WORLD, ret); \ - } \ -}while(0) - -#define KB 1024 -#define MB 1048576 - -#define PRINT(...) do{printf(__VA_ARGS__); fflush(stdout);}while(0) -#define TRACE(...) do{PRINT("%s:%d[%s]", __FILE__, __LINE__, __FUNCTION__); PRINT(__VA_ARGS__);}while(0) -#define TRACE_IF(cond,...) do{if((cond)) {TRACE(__VA_ARGS__);}}while(0) - -#define MAX_NB_SERVERS 256 /* For I'm using byte offsets and writing/reading char to check the test results. - I'm writing ranks to know who wrote where and who read from where; so - more than 255 is troublesome. Writing char is just for the readability - of the results of this test. - */ - - -#define DEFAULT_PORT_BASE 5000 -uint16_t port_base = DEFAULT_PORT_BASE; - -#define MAX_DATA_SIZE 50*MB -int rank = -1; -int size = -1; -int nb_servers = -1; - -triton_addr_t self; -char str_self[51]; //to containg for instance "mpi://0" for rank 0 -triton_msg_group_t group; -triton_msg_tag_t tag; -triton_addr_t server_addrs[MAX_NB_SERVERS]; - -//TODO: Fix linked lists to use the triton ones - -typedef struct node_link_t -{ - struct node_link_t* next; -}node_link_t; - -static void add_node(void** node_head, void** node_tail, void* node) -{ - node_link_t **head = (node_link_t**)(node_head); - node_link_t **tail = (node_link_t**)(node_tail); - if(*tail) - { - (*tail)->next = (node_link_t*)node; - } - else - { - *head = (node_link_t*)node; - } - *tail = (node_link_t*)node; -} - -static void remove_node(void** node_head, void** node_tail, void* node) -{ - node_link_t *cur = (node_link_t*)(*node_head), *previous = NULL; - while(cur) - { - if(cur == node) - { - if(previous) - previous->next = cur->next; - else - *node_head = cur->next; - if(*node_tail == cur) - *node_tail = previous; - break; - } - previous = cur; - cur = cur->next; - } -} - -static int is_prime(int n) -{ - int i, half = n/2; - if(n == 0) - return 0; - if(n == 1) - return 1; - for(i=2; i<=half; i++) - if( (n%i) == 0) - return 0; - return 1; -} - -static inline int is_server(int process_rank) -{ - if(size <=2 ) - return process_rank >=size -1; - return is_prime(process_rank); -} - - -/*Just some arbitrary logic to decide who is client and who is server -*/ -static inline int is_local_process_a_server() -{ - return is_server(rank); -} - - -char ip_file_name[151] = {0}; - - -typedef struct ip_str_t -{ - char ip[51]; -}ip_str_t; - -ip_str_t *str_ip_addresses; - -/*Build the IP addresses -*/ -static triton_ret_t build_ips() -{ - int i; - str_ip_addresses = malloc(rank*sizeof(ip_str_t)); - if(!str_ip_addresses) - return TRITON_ERR_NOMEM; - if(ip_file_name[0]) - { - FILE* ip_file = fopen(ip_file_name, "r"); - if(!ip_file) - { - PRINT("[WARNING!] Failed to open ip_file\n"); - goto use_loopback; //fallback to using loopback. - //This means that if all the involved processes - //are not in the same node, the job will fail - } - for(i=0; i<size; i++) - (void)fgets(str_ip_addresses[i].ip, 51, ip_file); - fclose(ip_file); - } - return TRITON_SUCCESS; -use_loopback: - for(i=0; i<size; i++) - strcpy(str_ip_addresses[i].ip, ETHERNET_LOOPBACK); - return TRITON_SUCCESS; -} - - -static void set_environment() -{ - int ret; - char env_string[101]; - char *env_value = NULL; - if((env_value = getenv(ENV_PORT_BASE)) != NULL) - { - int val = atoi(env_value); - if(val) - port_base = (uint16_t) val; - } - sprintf(env_string, "%s=%s", str_ip_addresses[rank].ip, str_ip_addresses[rank].ip); - ret = putenv(env_string); - triton_assert(ret == 0); - sprintf(env_string, "%s=%d", ENV_ETHERNET_PORT, port_base+rank); - ret = putenv(env_string); - triton_assert(ret == 0); -} - -static __blocking void open_endpoints() -{ - triton_string_t tstring; - self = triton_addr_self("ssm"); - triton_addr_to_string(self, &tstring); - strcpy(str_self, tstring.string); - triton_msg_group_add("ssm", "test-one-sided", &group); - tag = triton_msg_new_tag(group); -} - -static __blocking void close_endpoints() -{ - int i; - triton_addr_free(self); - for(i=0; i< nb_servers; i++) - triton_addr_free(server_addrs[i]); -} - - - -static __blocking void discover_servers(void) -{ - triton_ret_t tret; - int ret; - int i; - nb_servers = 0; - char str_addr[51]; - for(i=0; i< size; i++) - { - if(is_server(i)) - { - sprintf(str_addr, "ssm://tcp::%s|%d:", str_ip_addresses[i].ip, port_base+i); - tret = triton_addr_lookup(str_addr, &server_addrs[nb_servers++]); - triton_assert(tret == TRITON_SUCCESS); - } - } -} - - -FILE* trace_file = NULL; -#define TRACE_IN_FILE(...) do{fprintf(trace_file, __VA_ARGS__);}while(0) - -static void init_trace_in_file() -{ - char file_name[100]; - sprintf(file_name, "trace_%s_rank_%d", is_local_process_a_server()?"server":"client", - rank); - trace_file = fopen(file_name, "w"); -} - -static void close_trace_in_file() -{ - fclose(trace_file); -} - - -char* server_data = NULL; -char* read_server_data = NULL; -char* write_server_data = NULL; -triton_mem_reg_handle_t server_mrh; - -static void init_server_data() -{ - triton_ret_t tret; - if(!server_data) - { - server_data = triton_buffer_allocate(self, MAX_DATA_SIZE); - triton_assert(server_data); - tret = triton_register_memory(self, server_data, MAX_DATA_SIZE, &server_mrh); - triton_assert(tret == TRITON_SUCCESS); - } - memset(server_data, rank, MAX_DATA_SIZE); - write_server_data = read_server_data = server_data; - //write_server_data = &server_data[SERVER_DATA_SIZE/2]; -} - -static void free_server_data() -{ - triton_ret_t tret; - if(server_data) - { - tret = triton_unregister_memory(self, server_mrh); - triton_assert(tret == TRITON_SUCCESS); - triton_buffer_free(self, server_data); - server_data = NULL; - } -} - -/* -iov_items are assigned round robin starting from the primarily contacted server ... -*/ -static inline int get_nb_responding_servers(triton_iov_item_t* iov, int *iov_indices, int nb_iov_items) -{ - return MIN(nb_servers, nb_iov_items); -} - - -/* -iov_items are assigned round robin starting from the primarily contacted server ... -Returns the number of iov the server is expected to service -*/ -static int get_response_map_for_specific_server(int nb_iov_items, int* iov_indices, - triton_addr_t server_addr) -{ - /*start at address of primarily contacted server to ensure that it gets at least 1*/ - - int i, k; - int next_server_index = -1; - for(i=0; i<nb_servers; i++) - if(triton_addr_equal(server_addrs[i], self)) - { - next_server_index = i; - break; - } - triton_assert(next_server_index != -1); - for(i=0, k=0; i<nb_iov_items; i++) - { - if(triton_addr_equal(server_addrs[next_server_index], server_addr)) - iov_indices[k++] = i; - next_server_index = (next_server_index+1)%nb_servers; - } - return k; -} - -#define TEST_TYPE_READ 0 -#define TEST_TYPE_WRITE 1 -#define TEST_TYPE_ACKS 2 -#define TEST_TYPE_READ_SCHEDULE_TRANSFER_REQUEST 4 -#define TEST_TYPE_WRITE_SCHEDULE_TRANSFER_REQUEST 6 -#define TEST_TYPE_ACK_FRAGMENT 7 - -#define TEST_TYPE_SHUTDOWN 10 - -#define MAX_RPC_SIZE 4*KB -#define MAX_NB_IOV_ITEMS 50 - -typedef enum io_type_t -{ - IOT_UNDEFINED, - IOT_READ, - IOT_WRITE -}io_type_t; - - -static __blocking void notify_for_client_shutdown() -{ - int i; - triton_ret_t tret; - char payload_content = TEST_TYPE_SHUTDOWN; - char* payload = &payload_content; - uint32_t packet_size = sizeof(char); - for(i=0; i<nb_servers; i++) - { - do - { - tret = triton_msg_send(server_addrs[i], group, tag, 1, &payload, &packet_size); - } - while(tret != TRITON_SUCCESS); - triton_assert(tret == TRITON_SUCCESS); - } -} - -/*Scan all the iov_items and compute an offset and size that will exactly include -them all. -out_mem_pub_params contains the offset and size of the required mem_pub_handle -*/ -static void get_io_address_extent(triton_iov_item_t* iovs, int nb_iov_items, - triton_iov_item_t* out_mem_pub_params) -{ - size_t smallest_start = iovs[0].offset, temp_start, largest_end = iovs[0].offset + iovs[0].size, temp_end; - int i; - for(i=1; i<nb_iov_items; i++) - { - temp_start = iovs[i].offset; - temp_end = temp_start + iovs[i].size; - smallest_start = MIN(smallest_start, temp_start); - largest_end = MAX(largest_end, temp_end); - } - out_mem_pub_params->offset = smallest_start; - out_mem_pub_params->size = largest_end - smallest_start; -} - -#if 0 -int *g_rpc_size; -char *g_rpc_packet; -triton_iov_item_t* g_iovs; -int g_nb_iov_items; -char* g_serialized_mem_pub; -char* g_str_client_addr; -uint64_t g_unique_token; -uint32_t *g_offset; -int g_serialized_mem_pub_size; -#endif - -/* -unique_token is used to identify a specific request. Two pairs (str_client_addr, unique_token) -will never exist at the same time. -*/ -static int build_io_rpc(triton_iov_item_t* iovs, int* valid_iov_indices, int nb_iov_items, - char* serialized_mem_pub, int serialized_mem_pub_size, char* str_client_addr, uint64_t unique_token, - char* rpc_packet) -{ - triton_iov_item_t partial_iov[MAX_NB_IOV_ITEMS]; - int i; - for(i=0; i<nb_iov_items; i++) - { - partial_iov[i].offset = iovs[valid_iov_indices[i]].offset; - partial_iov[i].size = iovs[valid_iov_indices[i]].size; - } - - int rpc_size = 0; - - strcpy(rpc_packet, str_client_addr); - rpc_size += (strlen(str_client_addr)+1); - - memcpy(rpc_packet + rpc_size, &unique_token, sizeof(uint64_t)); - rpc_size+=sizeof(uint64_t); - - memcpy(rpc_packet + rpc_size, &serialized_mem_pub_size, sizeof(int)); - rpc_size+=sizeof(int); - - memcpy(rpc_packet + rpc_size, serialized_mem_pub, serialized_mem_pub_size); - rpc_size += serialized_mem_pub_size; - - memcpy(rpc_packet + rpc_size, &nb_iov_items, sizeof(nb_iov_items)); - rpc_size += sizeof(nb_iov_items); - - memcpy(rpc_packet + rpc_size, partial_iov, sizeof(triton_iov_item_t)*nb_iov_items); - rpc_size += sizeof(triton_iov_item_t)*nb_iov_items; - - return rpc_size; -} - - -static void extract_io_rpc_data(char* rpc_packet, triton_iov_item_t** iovs, int *nb_iov_items, - int *serialized_mem_pub_size, char** serialized_mem_pub, char **str_client_addr, uint64_t *unique_token) -{ - uint32_t offset = 0; - - *str_client_addr = rpc_packet; - offset += (strlen(*str_client_addr) + 1); - - *unique_token = *(uint64_t*)(rpc_packet + offset); - offset += sizeof(uint64_t); - - - *serialized_mem_pub_size = *((int*)(rpc_packet + offset)); - offset += sizeof(int); - - *serialized_mem_pub = (char*)(rpc_packet + offset); - offset += *serialized_mem_pub_size; - - *nb_iov_items = *((int*)(rpc_packet + offset)); - offset += sizeof(int); - - *iovs = (triton_iov_item_t*)(rpc_packet + offset); -} - - -static __blocking void do_client_io(char* mem_start, triton_mem_reg_handle_t mrh, triton_iov_item_t* iovs, - int nb_iov_items, io_type_t iot) -{ - triton_iov_item_t mem_pub_handle_params; - triton_addr_t primary_server_addr = server_addrs[rand()%nb_servers]; - int ret; - int i; - - triton_ret_t tret; - triton_mem_pub_handle_t mph; - MPI_Status status; - char some_buf[MAX_RPC_SIZE]; - char rpc_packet[MAX_RPC_SIZE]; //we're not really doing RPC here but this packet will contain approximately - //the same data as the equivalent RPC payload - char* packet_ptr = NULL; //just to make gcc happy; see where it is used - int serialized_mem_pub_h_size; - uint32_t rpc_size = 0; - uint32_t byte_received = 0; - - get_io_address_extent(iovs, nb_iov_items, &mem_pub_handle_params); - - int *valid_iov_indices = malloc(nb_iov_items*sizeof(int)); - for(i=0; i<nb_iov_items; i++) - valid_iov_indices[i] = i; - - tret = triton_publish_memory(self, mrh, mem_pub_handle_params.offset, mem_pub_handle_params.size, &mph); - triton_assert(tret == TRITON_SUCCESS); - - serialized_mem_pub_h_size = triton_get_serialized_handle_size(self, MEM_PUB_HANDLE_TYPE, &mph); - tret = triton_get_serialized_handle(self, MEM_PUB_HANDLE_TYPE, (void*)&mph, some_buf); - triton_assert(tret == TRITON_SUCCESS); - - rpc_size = build_io_rpc(iovs, valid_iov_indices, nb_iov_items, some_buf, serialized_mem_pub_h_size, - str_self, (uint64_t)mph, &rpc_packet[1]); - free(valid_iov_indices); - rpc_packet[0] = iot == IOT_READ ? TEST_TYPE_READ : TEST_TYPE_WRITE; - rpc_size++; - - //issue request - packet_ptr = rpc_packet; - PRINT("[CLIENT %s] about to issue %s request %lu\n", str_self, iot == IOT_READ ?"READ":"WRITE", - (uint64_t)mph); - - do - { - tret = triton_msg_send(primary_server_addr, group, tag, 1, &packet_ptr, &rpc_size); - } - while(tret != TRITON_SUCCESS); //retries - triton_assert(tret == TRITON_SUCCESS); - - //receive ack - tret = triton_msg_recv(primary_server_addr, group, tag, 1, &packet_ptr, &rpc_size, &byte_received); - triton_assert(tret == TRITON_SUCCESS); - triton_assert(rpc_packet[0] == TEST_TYPE_ACKS); - - tret = triton_wait(self, mph, (void*)(rpc_packet+sizeof(char))); - triton_assert(tret == TRITON_SUCCESS); - - tret = triton_unpublish_memory(self, mph); - triton_assert(tret == TRITON_SUCCESS); - - if(iot == IOT_READ) - { - //Display the whole region covered by the mem_pub_handle - PRINT("[CLIENT %s] DONE READING STUFF \n", str_self); - for(i=mem_pub_handle_params.offset; i<mem_pub_handle_params.size+mem_pub_handle_params.offset; i++) - TRACE_IN_FILE("%d: %d\n", i, mem_start[i]); - TRACE_IN_FILE("\n\n"); - } - else - { - TRACE_IN_FILE("[CLIENT %s] DONE WRITING STUFF ", str_self); - PRINT("[CLIENT %s] DONE WRITING STUFF \n", str_self); - } - - TRACE_IN_FILE("\n"); -} - - -FILE* input_file = NULL; -char input_file_name[201]; - -static void open_input() -{ - input_file = fopen(input_file_name, "r"); - triton_assert(input_file); -} - -static void close_input() -{ - fclose(input_file); -} - - -/* -Returns 1 if this process has iov. 0 otherwise. -If this process has iovs, then the file read pointer is set -to the begining of its iovs -*/ -static int has_iov() -{ - char buf[1025]; - char str_rank[5]; - int addr_length; - sprintf(str_rank, "%d", rank); - addr_length = strlen(str_rank); - while(fgets(buf, 1024, input_file)) - { - if((strlen(buf) > addr_length + 2) && buf[0] == '<' && strncmp(buf+1, str_rank, addr_length) == 0) - return 1; - } - return 0; -} - - -static void extract_iov_item(char* buf, triton_iov_item_t* iov_item) -{ - int i; - char *token = NULL; - int j=0; - for(j=0; j<2; j++) - { - token = strtok(j?NULL:buf, ","); - while(!isdigit(*token)) - token++; - i = strlen(token); - i--; - while(!isdigit(token[i])) - i--; - token[i+1] = '\0'; - if(j == 0) - iov_item->offset = (uint64_t)atol(token); - else - iov_item->size = (uint64_t)atol(token); - } -} - -typedef enum input_extract_phase_t -{ - UNDEFINED, - NB_IOV_ITEMS, - IOV_ITEMS -}input_extract_phase_t; - -/* -Extract a single iov for a single operation (READ or WRITE) -If iot == IOT_UNDEFINED at the exit of the function, then no more to read. -*/ -static int extract_iov(triton_iov_item_t** iov, io_type_t* iot) -{ - char buf[1024]; - char *line = buf; - char end_of_iov[51]; - int nb_iov_items = 0; - input_extract_phase_t input_extract_phase = UNDEFINED; - int nb_extracted_iov_items = 0; - - sprintf(end_of_iov, "</%d>", rank); - *iot = IOT_UNDEFINED; - while(fgets(line, 1024, input_file)) - { - while(*line == ' ' || *line == '\t') - line++; - if(*line == '\n' || *line == '#') - continue; - if(strncmp(line, end_of_iov, strlen(end_of_iov)) == 0) - return nb_extracted_iov_items; - if(strncmp(line, "<READ>", strlen("<READ>")) == 0) - { - *iot = IOT_READ; - input_extract_phase = NB_IOV_ITEMS; - continue; - } - else if(strncmp(line, "<WRITE>", strlen("<WRITE>")) == 0) - { - *iot = IOT_WRITE; - input_extract_phase = NB_IOV_ITEMS; - continue; - } - else if(strncmp(line, "</READ>", strlen("</READ>")) == 0 || - strncmp(line, "</WRITE>", strlen("</WRITE>")) == 0) - return nb_extracted_iov_items; - if(isdigit(line[0])) - { - buf[strlen(buf)-1]='\0'; - if(input_extract_phase == NB_IOV_ITEMS) - { - nb_iov_items = atoi(buf); - *iov = malloc(nb_iov_items * sizeof(triton_iov_item_t)); - triton_assert(*iov); - input_extract_phase = IOV_ITEMS; - } - else if(input_extract_phase == IOV_ITEMS) - { - extract_iov_item(buf, &((*iov)[nb_extracted_iov_items++])); - } - } - } - return nb_extracted_iov_items; -} - -/* -*/ -static __blocking void enter_client_loop() -{ - char *buffer = NULL; - triton_ret_t tret; - triton_mem_reg_handle_t mrh1; - io_type_t iot; - int nb_iov_items; - - buffer = malloc(MAX_DATA_SIZE*sizeof(char)); - triton_assert(buffer); - - triton_iov_item_t *iov = NULL; - - memset(buffer, rank, MAX_DATA_SIZE); //initialize the client memory - - tret = triton_register_memory(self, buffer, MB, &mrh1); - triton_assert(tret == TRITON_SUCCESS); - - pwait - { - pprivate triton_iov_item_t* iov_for_a_pbranch; - if(has_iov()) - { - /*There is a bit of a bad practice here. The iov is allocated in extract_iov - and freed in the pbranch. - */ - nb_iov_items = extract_iov(&iov, &iot); - while( nb_iov_items ) - { - iov_for_a_pbranch = iov; //this because the pointer will be lost in the - //next extract_iov call - pbranch - { - do_client_io(buffer, mrh1, iov_for_a_pbranch, nb_iov_items, iot); - free(iov_for_a_pbranch); - } - nb_iov_items = extract_iov(&iov, &iot); - } - } - } - - tret = triton_unregister_memory(self, mrh1); - triton_assert(tret == TRITON_SUCCESS); - - notify_for_client_shutdown(); - free(buffer); -} - -static void display_server_side_read_data(char* buf_start, triton_iov_item_t *iov, int nb_iov_items, - char* str_client_addr) -{ - int i; - triton_iov_item_t data_span; - get_io_address_extent(iov, nb_iov_items, &data_span); - TRACE_IN_FILE("<-----TRACING MEMORY AFTER I/O WITH CLIENT %s------->\n", str_client_addr); - for(i=data_span.offset; i<data_span.offset+data_span.size; i++) - { - TRACE_IN_FILE("%d:%d\n", i, buf_start[i]); - } - TRACE_IN_FILE("</-----TRACING MEMORY AFTER I/O WITH CLIENT %s------->\n\n", str_client_addr); -} - -static __blocking void do_server_io(triton_addr_t client_addr, triton_mem_pub_handle_t mem_pub_handle, - triton_iov_item_t *iov, int nb_iov_items, int8_t test_type) -{ - int i; - triton_ret_t tret; - - char* buf_start = test_type == TEST_TYPE_READ? read_server_data: write_server_data; - char **bufs = malloc(sizeof(char*)*nb_iov_items); - triton_assert(bufs); - for(i=0; i<nb_iov_items; i++) - bufs[i] = (char*)(buf_start + iov[i].offset); - - if(test_type == TEST_TYPE_READ) - { - tret = triton_msg_put(self, client_addr, mem_pub_handle,iov, bufs, nb_iov_items); - } - else - { - tret = triton_msg_get(self, client_addr, mem_pub_handle, iov, bufs, nb_iov_items); - } - triton_assert(tret == TRITON_SUCCESS); - free(bufs); -} - -/* -Wait infos are kept by the primarily contacted server. -Each secondarily contacted server sends its ack to the -primarily contacted server with a unique token that identifies the -request the ack is being sent for. -nb_expected_acks_from_other_servers is decremented by the primary server upon each reception -of unique_token until it reaches 0. Then the primarily contacted server sends the final ack to the -client and releases the wait info object -*/ -typedef struct server_side_wait_info_t -{ - struct server_side_wait_info_t *next; - triton_addr_t client_addr; - int nb_expected_acks_from_other_servers; - uint64_t unique_token; -}server_side_wait_info_t; - - -server_side_wait_info_t* wait_info_head = NULL; -server_side_wait_info_t* wait_info_tail = NULL; - - -static void add_wait_info(triton_addr_t client_addr, int nb_expected_acks_from_other_servers, uint64_t unique_token) -{ - server_side_wait_info_t* wait_info = malloc(sizeof(server_side_wait_info_t)); - triton_assert(wait_info); - wait_info->client_addr = client_addr; - wait_info->next = NULL; - wait_info->nb_expected_acks_from_other_servers = nb_expected_acks_from_other_servers; - wait_info->unique_token = unique_token; - add_node((void**)(&wait_info_head), (void**)(&wait_info_tail), (void*)wait_info); -} - -static server_side_wait_info_t* retrieve_wait_info(uint64_t unique_token) -{ - server_side_wait_info_t* wait_info = wait_info_head; - while(wait_info) - { - if(wait_info->unique_token == unique_token) - return wait_info; - wait_info = wait_info->next; - } - return NULL; -} - -static inline void remove_wait_info(server_side_wait_info_t* wait_info) -{ - remove_node((void**)(&wait_info_head), (void**)(&wait_info_tail), (void*)wait_info); - free(wait_info); -} - -static __blocking void enter_server_loop() -{ - int nb_client_shutdowns = 0; - volatile int clear_to_read_next_incoming_rpc = 1; - volatile int request_is_received; - triton_mutex_t mutex; - triton_mutex_t rpc_mutex; - char listener_rpc_packet[MAX_RPC_SIZE]; - char* listener_packet_ptr = listener_rpc_packet; - - triton_mutex_init(&mutex, NULL); //to protect the shutdown variable and possibly other stuff - triton_mutex_init(&rpc_mutex, NULL); //to protect the shutdown variable and possibly other stuff - - pwait - { - pprivate int serialized_mem_pub_h_size; - pprivate char rpc_packet[MAX_RPC_SIZE]; - char* packet_ptr = rpc_packet; //just to make gcc happy - pprivate triton_iov_item_t* iovs; - pprivate char* serialized_mem_pub; - pprivate triton_addr_t client_addr; - pprivate char* str_client_addr; - pprivate uint32_t rpc_size; - pprivate uint64_t unique_token; //used to differentiate requests from the same client. - - pprivate int iov_indices[MAX_NB_IOV_ITEMS]; - pprivate char io_rpc_packets[MAX_NB_SERVERS][MAX_RPC_SIZE]; - pprivate int ret; - pprivate triton_ret_t tret; - pprivate triton_mem_pub_handle_t mem_pub_handle; - pprivate int nb_scheduled_servers = 0; - pprivate int i; - pprivate int j; - pprivate int nb_iov_items; - pprivate int nb_iov_items_for_specific_server; - pprivate server_side_wait_info_t* wait_info; - triton_iov_item_t partial_iov[MAX_NB_IOV_ITEMS]; - - triton_addr_t requester_addr; - - pprivate triton_addr_t private_requester_addr; - pprivate triton_msg_tag_t unused_tag; - pprivate uint32_t count; - pprivate uint32_t max_receive_size = MAX_RPC_SIZE; - pprivate uint32_t received_bytes; - - for(;;) - { - if(nb_client_shutdowns == size - nb_servers) - { - PRINT("Server %s is shutting down. Hasta luego!\n", str_self); - break; - } - - /* - The server waits for the last spawned pbranch to copy the rpc data before it goes - on listening to the next one again. The wait is necessary since the same - buffer is used. - */ - while(!clear_to_read_next_incoming_rpc); /*might use sleep or a semaphore, etc. instead here ... - ... thinking that the wait will usually be short. - (see below at the begining of the pbracnh)*/ - - clear_to_read_next_incoming_rpc = 0; - request_is_received = 0; - pbranch - { - tret = triton_msg_recv_any(group, &requester_addr, &unused_tag, 1, - &listener_packet_ptr, &max_receive_size, &received_bytes); - triton_assert(tret == TRITON_SUCCESS); - request_is_received = 1; - } - while(!request_is_received) - usleep(1000); - request_is_received = 0; - - pbranch - { - memcpy(rpc_packet, listener_rpc_packet, MAX_RPC_SIZE); - memcpy(&private_requester_addr, &requester_addr, sizeof(triton_addr_t)); - clear_to_read_next_incoming_rpc = 1; - - switch(rpc_packet[0]) - { - case TEST_TYPE_READ: - case TEST_TYPE_WRITE: - extract_io_rpc_data(&rpc_packet[1], &iovs, &nb_iov_items, &serialized_mem_pub_h_size, &serialized_mem_pub, - &str_client_addr, &unique_token); - PRINT("Primary Server %s handling I/O %lu\n", str_self, unique_token); - client_addr = private_requester_addr; - nb_scheduled_servers = get_nb_responding_servers(iovs, iov_indices, nb_iov_items); - if(nb_scheduled_servers > 1) - add_wait_info(client_addr, nb_scheduled_servers - 1, unique_token); - - /* - We adopt the following simple response map strategy. Starting from itself, - this primarily contacted server assigns each iov_item to the next server address (cycling) - until all the iov_items are assigned. - */ - for(i=0, j=0; nb_scheduled_servers > 1 && i<nb_servers; i++) - { - char* io_packet_ptr; - if(triton_addr_equal(server_addrs[i], self)) - continue; //handle own stuff in last position - nb_iov_items_for_specific_server = get_response_map_for_specific_server( - nb_iov_items, iov_indices, server_addrs[i]); - if(nb_iov_items_for_specific_server == 0) - continue; //the response map does not concern server_addrs[i] - rpc_size = build_io_rpc(iovs, iov_indices, nb_iov_items_for_specific_server, - serialized_mem_pub, serialized_mem_pub_h_size, str_client_addr, - unique_token, &io_rpc_packets[j][1]); - io_rpc_packets[j][0] = rpc_packet[0] == TEST_TYPE_READ? - TEST_TYPE_READ_SCHEDULE_TRANSFER_REQUEST: - TEST_TYPE_WRITE_SCHEDULE_TRANSFER_REQUEST; - rpc_size++; - io_packet_ptr = io_rpc_packets[j]; - do - { - tret = triton_msg_send(server_addrs[i], group, tag, 1, &io_packet_ptr, &rpc_size); - } - while(tret != TRITON_SUCCESS); - triton_assert(tret == TRITON_SUCCESS); - j++; - } - - /*This server sends its share of responses now*/ - tret = triton_create_handle_from_serialization(self, MEM_PUB_HANDLE_TYPE, (void*)(&mem_pub_handle), - serialized_mem_pub); - triton_assert(tret == TRITON_SUCCESS); - nb_iov_items_for_specific_server = get_response_map_for_specific_server( - nb_iov_items, iov_indices, self); - for(i=0; i<nb_iov_items_for_specific_server; i++) - { - partial_iov[i].offset = iovs[iov_indices[i]].offset; - partial_iov[i].size = iovs[iov_indices[i]].size; - } - PRINT("Primary Server %s about to serve I/O %lu\n", - str_self, unique_token); - tret = triton_start_epoch(self, mem_pub_handle); - triton_assert(tret == TRITON_SUCCESS); - do_server_io(client_addr, mem_pub_handle, partial_iov, - nb_iov_items_for_specific_server, rpc_packet[0] /*READ or WRITE*/); - tret = triton_end_epoch(self, mem_pub_handle); - triton_assert(tret == TRITON_SUCCESS); - PRINT("Primary Server %s done serving I/O %lu\n", - str_self, unique_token); - if(rpc_packet[0] == TEST_TYPE_WRITE) - { - display_server_side_read_data(server_data, partial_iov, - nb_iov_items_for_specific_server, str_client_addr); - } - tret = triton_free_serialized_handle(self, MEM_PUB_HANDLE_TYPE, - (void*)(&mem_pub_handle)); - triton_assert(tret == TRITON_SUCCESS); - if(nb_scheduled_servers == 1) - { - //send ack to client right away - rpc_packet[0]=TEST_TYPE_ACKS; - memcpy(&rpc_packet[1], &unique_token, sizeof(unique_token)); - rpc_size = 1 + sizeof(unique_token); - PRINT("Primary Server %s about to acknowledge I/O %lu\n", - str_self, unique_token); - packet_ptr = rpc_packet; - do - { - tret = triton_msg_send(client_addr, group, tag, 1, - &packet_ptr, &rpc_size); - } - while(tret != TRITON_SUCCESS); - triton_assert(tret == TRITON_SUCCESS); - PRINT("Primary Server %s completed and acknowledged I/O %lu\n", - str_self, unique_token); - } - break; - - //Handle pipeline response requests from some primarily contacted server - case TEST_TYPE_READ_SCHEDULE_TRANSFER_REQUEST: - case TEST_TYPE_WRITE_SCHEDULE_TRANSFER_REQUEST: - extract_io_rpc_data(&rpc_packet[1], &iovs, &nb_iov_items, &serialized_mem_pub_h_size, &serialized_mem_pub, - &str_client_addr, &unique_token); - PRINT("Auxiliary Server %s received pipelined request for I/O %lu\n", - str_self, unique_token); - - tret = triton_create_handle_from_serialization(self, MEM_PUB_HANDLE_TYPE, (void*)(&mem_pub_handle), - serialized_mem_pub); - triton_assert(tret == TRITON_SUCCESS); - tret = triton_addr_lookup(str_client_addr, &client_addr); - triton_assert(tret); - tret = triton_start_epoch(self, mem_pub_handle); - triton_assert(tret == TRITON_SUCCESS); - do_server_io(client_addr, mem_pub_handle, iovs, nb_iov_items, - rpc_packet[0] == TEST_TYPE_READ_SCHEDULE_TRANSFER_REQUEST? - TEST_TYPE_READ:TEST_TYPE_WRITE); - tret = triton_end_epoch(self, mem_pub_handle); - triton_assert(tret == TRITON_SUCCESS); - if(rpc_packet[0] == TEST_TYPE_WRITE_SCHEDULE_TRANSFER_REQUEST) - { - display_server_side_read_data(server_data, iovs, nb_iov_items, str_client_addr); - } - tret = triton_free_serialized_handle(self, MEM_PUB_HANDLE_TYPE, - (void*)(&mem_pub_handle)); - triton_assert(tret == TRITON_SUCCESS); - rpc_packet[0]=TEST_TYPE_ACK_FRAGMENT; - memcpy(&rpc_packet[1], &unique_token, sizeof(unique_token)); - rpc_size = 1 + sizeof(unique_token); - packet_ptr = rpc_packet; - do - { - tret = triton_msg_send(private_requester_addr/*primary server in this case*/, group, - tag, 1, &packet_ptr, &rpc_size); - } - while(tret != TRITON_SUCCESS); - triton_assert(tret == TRITON_SUCCESS); - triton_addr_free(client_addr); - break; - - /*Below is primarily contacted server receiving "ack fragments" from other - contacted servers for pipelining*/ - case TEST_TYPE_ACK_FRAGMENT: - unique_token = *((uint64_t*)(&rpc_packet[1])); - triton_mutex_lock(&mutex); - wait_info = retrieve_wait_info(unique_token); - triton_mutex_unlock(&mutex); - triton_assert(wait_info); - if(--wait_info->nb_expected_acks_from_other_servers == 0) - { - rpc_packet[0]=TEST_TYPE_ACKS; - memcpy(&rpc_packet[1], &unique_token, sizeof(unique_token)); - rpc_size = 1 + sizeof(unique_token); - packet_ptr = rpc_packet; - do - { - tret = triton_msg_send(wait_info->client_addr, group, tag, 1, &packet_ptr, &rpc_size); - } - while(tret != TRITON_SUCCESS); - triton_assert(tret == TRITON_SUCCESS); - PRINT("Primary Server %s acknowledging I/O %lu after pipelined transfers\n", - str_self, unique_token); - - triton_mutex_lock(&mutex); - remove_wait_info(wait_info); - triton_mutex_unlock(&mutex); - } - break; - - case TEST_TYPE_SHUTDOWN: - triton_mutex_lock(&mutex); - nb_client_shutdowns++; - PRINT("Server %s received client shutdown request no %d\n", str_self, - nb_client_shutdowns); - triton_mutex_unlock(&mutex); - break; - default: - PRINT("Server %s What the heck ...\n", str_self); - triton_assert(0); - } - } //pbranch - } //for - } //end pwait - triton_mutex_destroy(&mutex); - triton_mutex_destroy(&rpc_mutex); -} - -static void print_test_description() -{ - if(rank != 0) - return; - printf("\n\n Usage is: prog_name input_file_name [IP_Addresses_file_name]\n" - "See README_FOR_ONE_SIDED in \x1b[34mtriton/code/src/net/tests/\x1b[0m.\n" - "A sample input file named sample_one_sided_test_input can be copied from the aforementioned location as well\n\n"); -} - -__blocking int entry_point(int argc, char** argv) -{ - int dummy; - triton_ret_t tret; - srand(time(NULL)); - MPI_Comm_rank(MPI_COMM_WORLD, &rank); - MPI_Comm_size(MPI_COMM_WORLD, &size); - - if(argc < 2) - { - if(rank == 0) - { - print_test_description(); - } - return 0; - } - if(size > 256) - { - if(rank == 0) - { - printf("This test modifies memory at byte granularity. In order to allow the complete visualization of\n" - "which process modified which other process' data, the written data are process ranks. We needed\n" - "to fit all the ranks in a byte. The maximum allowed job size is therefore 256. The net-module \n" - "can handle virtually any job size. This limit of 256 concerns only the test and is for easiness of \n" - "interpretation only"); - } - return 0; - } - if(argc == 3) - strcpy(ip_file_name, argv[2]); - JZ_WAIT_FOR_DEBUGGER(); - if(build_ips() != TRITON_SUCCESS) - { - PRINT("Aborting for failure to get IP addreses"); - MPI_Abort(MPI_COMM_WORLD, 0); - } - set_environment(); - - - tret = triton_msg_ssm_init(); - triton_assert(tret == TRITON_SUCCESS); - strcpy(input_file_name, argv[1]); - open_endpoints(); - open_input(); - - if(size == 1) - { - printf("\nHello I'm a jobless server ... who's going to bug me if there is no client, hein?\n"); - return 0; - } - discover_servers(); - init_trace_in_file(); - if(!is_local_process_a_server()) - { - enter_client_loop(); - } - else - { - init_server_data(); - enter_server_loop(); - free_server_data(); - } - close_trace_in_file(); - close_input(); - close_endpoints(); - triton_msg_ssm_finalize(); - return 0; -} - -aesop_main_set(entry_point); diff --git a/code/src/net/tests/ssm-sr-test.ae b/code/src/net/tests/ssm-sr-test.ae deleted file mode 100644 index 05fb450..0000000 --- a/code/src/net/tests/ssm-sr-test.ae +++ /dev/null @@ -1,247 +0,0 @@ -#include "src/net/ssm/ssm-method.h" - -#include <aesop/aesop.h> -#include <mpi.h> -#include <stdio.h> -#include <errno.h> -#include "../triton-message-method.hae" -#include <time.h> -#include <stdlib.h> -#include <string.h> -#include <stdarg.h> -#include <ctype.h> -#include <unistd.h> -#include <stdlib.h> - - -#define JZ_DEBUG -static __attribute__((noinline,unused)) void __jz_wait_for_debugger(const char* file, const char* function, int line, int rank, int once) -{ - static int count = 0; - - if(count && once) - return; - count++; - printf("process ( pid = %d | rank = %d ) is waiting in %s at %s:%d for debugger\n", - getpid(), rank, function, file, line); - fflush(stdout); - for(;;); -} - -#ifdef JZ_DEBUG - #define JZ_WAIT_FOR_DEBUGGER() __jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 0) - #define JZ_WAIT_FOR_DEBUGGER_IF(cond) \ - do{if((cond))__jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 0);}while(0) - #define JZ_WAIT_ONCE_FOR_DEBUGGER() __jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 1) - #define JZ_WAIT_ONCE_FOR_DEBUGGER_IF(cond) \ - do{if((cond))__jz_wait_for_debugger(__FILE__, __FUNCTION__, __LINE__, -1, 1);}while(0) -#else - #define JZ_WAIT_FOR_DEBUGGER() - #define JZ_WAIT_FOR_DEBUGGER_IF() - #define JZ_WAIT_ONCE_FOR_DEBUGGER() - #define JZ_WAIT_ONCE_FOR_DEBUGGER_IF() -#endif - -#define ENV_ETHERNET_IP "ETHERNET_IP" -#define ENV_ETHERNET_PORT "ETHERNET_PORT" -#define ETHERNET_LOOPBACK "127.0.0.1" - -#define MIN(a,b) ((a)<=(b)?(a):(b)) -#define MAX(a,b) ((a)>=(b)?(a):(b)) -#define ABORT_ON_MPI_FAILURE(ret,err_msg_format,...) \ -do \ -{ \ - if((ret) != MPI_SUCCESS) \ - { \ - triton_err(triton_log_default, "%s:%d: [MPI Proc rank %d]" err_msg_format, \ - __FILE__, __LINE__, rank, ##__VA_ARGS__); \ - MPI_Abort(MPI_COMM_WORLD, ret); \ - } \ -}while(0) - -#define KB 1024 -#define MB 1048576 - -#define PRINT(...) do{printf(__VA_ARGS__); fflush(stdout);}while(0) -#define TRACE PRINT -#define TRACE_IF(cond,...) do{if((cond)) {PRINT("%s:%d[%s]", __FILE__, __LINE__, __FUNCTION__); PRINT(__VA_ARGS__);}}while(0) - -#define MAX_NB_SERVERS 256 /* For I'm using byte offsets and writing/reading char to check the test results. - I'm writing ranks to know who wrote where and who read from where; so - more than 255 is troublesome. Writing char is just for the readability - of the results of this test. - */ - - -#define DEFAULT_PORT_BASE 5000 - -#define MAX_DATA_SIZE 50*MB -int rank = -1; -int size = -1; -int nb_servers = -1; - -triton_addr_t self; -char str_self[51]; //to containg for instance "mpi://0" for rank 0 -triton_msg_group_t group; -triton_msg_tag_t tag; -triton_addr_t server_addrs[MAX_NB_SERVERS]; - -uint16_t port_base = DEFAULT_PORT_BASE; - -static int is_prime(int n) -{ - int i, half = n/2; - if(n == 0) - return 0; - if(n == 1) - return 1; - for(i=2; i<=half; i++) - if( (n%i) == 0) - return 0; - return 1; -} - -static inline int is_server(int process_rank) -{ - if(size <=2 ) - return process_rank >=size -1; - return is_prime(process_rank); -} - - -/*Just some arbitrary logic to decide who is client and who is server -*/ -static inline int is_local_process_a_server() -{ - return is_server(rank); -} - -static void set_environment() -{ - int ret; - char env_string[101]; - sprintf(env_string, "%s=%s", ENV_ETHERNET_IP, ETHERNET_LOOPBACK); - ret = putenv(env_string); - triton_assert(ret == 0); - sprintf(env_string, "%s=%d", ENV_ETHERNET_PORT, port_base+rank); - ret = putenv(env_string); - triton_assert(ret == 0); -} - -static __blocking void open_endpoints() -{ - triton_string_t tstring; - self = triton_addr_self("ssm"); - triton_addr_to_string(self, &tstring); - strcpy(str_self, tstring.string); - triton_msg_group_add("ssm", "test-one-sided", &group); - tag = triton_msg_new_tag(group); -} - -static __blocking void close_endpoints() -{ - int i; - triton_addr_free(self); - for(i=0; i< nb_servers; i++) - triton_addr_free(server_addrs[i]); -} - -/* -Discovering servers could be implemented differently; e.g. having the -processes read from a file who's who -*/ -static __blocking void discover_servers(void) -{ - triton_ret_t tret; - int ret; - int i; - nb_servers = 0; - char str_addr[51]; - for(i=0; i< size; i++) - { - if(is_server(i)) - { - sprintf(str_addr, "ssm://tcp::%s|%d", ETHERNET_LOOPBACK, port_base+i); - tret = triton_addr_lookup(str_addr, &server_addrs[nb_servers++]); - triton_assert(tret == TRITON_SUCCESS); - } - } -} - - -char buffer[MB]; -#define DATA_SIZE 100 - - -__blocking int entry_point(int argc, char** argv) -{ - int dummy; - int i; - triton_addr_t requester_addr; - triton_msg_tag_t unused_tag; - uint32_t max_recv_size; - uint32_t received_bytes; - char* buf = buffer; - - uint32_t data_size = DATA_SIZE; - triton_ret_t tret; - srand(time(NULL)); - MPI_Comm_rank(MPI_COMM_WORLD, &rank); - MPI_Comm_size(MPI_COMM_WORLD, &size); - - if(argc ==2) - { - i = atoi(argv[1]); - if(i) - port_base = (uint16_t)i; - } - set_environment(); - - - tret = triton_msg_ssm_init(); - triton_assert(tret == TRITON_SUCCESS); - open_endpoints(); - discover_servers(); - - for(i=0; i<DATA_SIZE; i++) - buffer[i] = rank; - - if(rank == 0) - { - sleep(1); - tret = triton_msg_send(server_addrs[0], group, tag, 1, &buf, &data_size); - triton_assert(tret == TRITON_SUCCESS) - for(i=0; i<DATA_SIZE; i++) - buffer[i] = rank+100; - sleep(2); - tret = triton_msg_send(server_addrs[0], group, tag, 1, &buf, &data_size); - triton_assert(tret != TRITON_SUCCESS) - } - else - { - tret = triton_msg_recv_any(group, &requester_addr, &unused_tag, 1, - &buf, &max_recv_size, &received_bytes); - triton_assert(tret == TRITON_SUCCESS); - TRACE("requester_addr = (l=%lx, u=%lx)\n", requester_addr.l, requester_addr.u); - TRACE("unused_tag = %u\n", unused_tag); - for(i=0; i<DATA_SIZE; i++) - printf("buffer[%d] = %d\n", i, buffer[i]); - /* - tret = triton_msg_recv_any(group, &requester_addr, &unused_tag, 1, - &buf, &max_recv_size, &received_bytes); - triton_assert(tret == TRITON_SUCCESS);*/ - tret = triton_msg_recv(requester_addr, group, unused_tag+1, 1, - &buf, &max_recv_size, &received_bytes); - triton_assert(tret == TRITON_SUCCESS); - for(i=0; i<DATA_SIZE; i++) - printf("buffer[%d] = %d\n", i, buffer[i]); - } - MPI_Barrier(MPI_COMM_WORLD); - //JZ_WAIT_FOR_DEBUGGER(); - - close_endpoints(); - triton_msg_ssm_finalize(); - return 0; -} - -aesop_main_set(entry_point); diff --git a/code/src/net/triton-addr.h b/code/src/net/triton-addr.h deleted file mode 100644 index 3493b80..0000000 --- a/code/src/net/triton-addr.h +++ /dev/null @@ -1,30 +0,0 @@ -#ifndef __TRITON_ADDR_H__ -#define __TRITON_ADDR_H__ - -#include <triton-string.h> -#include "src/common/triton-types.h" - -/** - * The triton_addr_t is an opaque object for addressing a node in the - * system. Internally, this address holds the messaging context - * (state for the method), and the method specific structures for the - * address. - */ -typedef uint128_t triton_addr_t; - -#define TRITON_ADDR_NULL_STATIC_INITIALIZER { .u = 0, .l = 0 } - -extern triton_addr_t triton_addr_null; -#define TRITON_ADDR_NULL triton_addr_null - -#include "src/common/triton-error.h" - -triton_ret_t triton_addr_to_string(triton_addr_t addr, triton_string_t *str); - -triton_ret_t triton_addr_lookup(const char *endpoint, triton_addr_t *addr); - -int triton_addr_equal(const triton_addr_t a1, const triton_addr_t a2); - -int triton_addr_is_null(const triton_addr_t a); - -#endif /* __TRITON_ADDR_H__ */ diff --git a/code/src/net/triton-message-method.hae b/code/src/net/triton-message-method.hae deleted file mode 100644 index d292d6a..0000000 --- a/code/src/net/triton-message-method.hae +++ /dev/null @@ -1,244 +0,0 @@ -#ifndef __TRITON_MESSAGE_METHOD_HAE__ -#define __TRITON_MESSAGE_METHOD_HAE__ - -#include "src/net/triton-addr.h" -#include "src/net/triton-message.hae" - -typedef uint64_t triton_msg_method_addr_t; -typedef uint32_t triton_msg_method_group_t; - - -struct triton_msg_method -{ - /** - * The name of this method. Must match the endpoint method prefix. e.g. endpoint=mpi://20, name=mpi - */ - const char *name; - - /** - * Get the local address. This is blocking, because some methods set the "self" address through the hints - * mechanism, which only gets propagated by blocking functions. - */ - __blocking triton_msg_method_addr_t (*addr_self)(void); - - int (*addr_equal)(triton_msg_method_addr_t a1, triton_msg_method_addr_t a2); - - /** - * Get the endpoint string for this address, *without* including the endpoint method prefix. - */ - triton_ret_t (*addr_to_string)(triton_msg_method_addr_t addr, triton_string_t *str); - - /** - * Lookup an address contained in a string. The format depends on the method type, but should - * not include the endpoint method prefix. - */ - triton_ret_t (*addr_lookup)(const char *name, triton_msg_method_addr_t *addr); - - /** - * Free an address - */ - triton_ret_t (*addr_free)(triton_msg_method_addr_t addr); - - /** - * Copy an address - */ - triton_ret_t (*addr_copy)(const triton_msg_method_addr_t orig_addr, triton_msg_method_addr_t *copy); - - /** - * Add a group with the given service name. - */ - triton_ret_t (*group_add)(const char *service_name, triton_msg_method_group_t *group); - - /** - * Get a new tag that can be used for matching sends and receives. - */ - triton_msg_tag_t (*new_tag)(triton_msg_method_group_t group); - - /** - * Send a message using a particular group and tag - */ - __blocking triton_ret_t (*send)(triton_msg_method_addr_t to, - triton_msg_method_group_t group, - triton_msg_tag_t tag, - int count, - char **buffers, - uint32_t *sizes); - - /** - * Receive (pre-posted) a message from a specific address, on a specific group, with a specific tag. - */ - __blocking triton_ret_t (*recv)(triton_msg_method_addr_t from, - triton_msg_method_group_t group, - triton_msg_tag_t tag, - int count, - char **buffers, - uint32_t *sizes, - uint32_t *bytes_received); - - /** - * Receive a message from a specific group, but from any address with any tag. - */ - __blocking triton_ret_t (*recv_any)(triton_msg_method_group_t group, - triton_msg_method_addr_t *from, - triton_msg_tag_t *tag, - int count, - char **buffers, - uint32_t *sizes, - uint32_t *bytes_received); - - /** - *Set the message size beyond which one-sided communication is used instead 2-sided for message transfers. - * The size is in bytes - */ - triton_ret_t (*set_one_sided_threshold)(size_t size); - - - /** - *Get the message size beyond which one-sided communication is used instead 2-sided for message transfers. - * The size is in bytes - */ - triton_ret_t (*get_one_sided_threshold)(size_t *size); - - /** - * Allocate a buffer for sending and receiving. Method can use this - * call to pin a buffer. - * TODO: Should we include information about the type of buffer to allocate? - */ - void * (*buffer_allocate)(size_t size); - - /** - * Free a buffer allocated with buffer_allocate. - */ - void (*buffer_free)(void *buffer); - - /** - * Register memory for the communication subsystem. - */ - triton_ret_t (*register_memory)(void* base, - size_t size, - triton_mem_reg_handle_t* handle); - /** - * Unregister memory. - */ - triton_ret_t (*unregister_memory)(triton_mem_reg_handle_t handle); - - /** - * Expose a registered memory portion to remote peers. - */ - __blocking triton_ret_t (*publish_memory)(triton_mem_reg_handle_t in_handle, - size_t offset, - size_t size, - triton_mem_pub_handle_t* out_handle); - - /** - * End access permissions to a memory portion previously exposed. - */ - __blocking triton_ret_t (*unpublish_memory)(triton_mem_pub_handle_t handle); - - - /*returns the size of buffer required to get the serialized representation - of the info type passed as argument - *pointer_to_handle must be compatible with handle_type - */ - int (*get_serialized_handle_size)(triton_serialized_handle_type_t handle_type, - void* pointer_to_handle); - - - /*Ask the net-module to provide the serialized net-module-specific info - associated with the handle *pointer_to_in_handle. - COMMENTS: - 1. pointer_to_in_handle must point to a handle compatible with handle_type - (e.g. if handle_type is MEM_PUB_HANDLE_TYPE, then pointer_to_in_handle must - point to a triton_mem_pub_handle_t). - - 2. serialized_handle must be large enough. A previous call to - get_serialized_handle_size is recommended to determine the required buffer size. - - 3. This function might succeed even if set_serialized_handle (below) was not called previously - to return pointer_to_in_handle. E.g. When publish_memory returns a handle h, any - subsequent call to get_serialized_handle with h and handle_type == MEM_PUB_HANDLE_TYPE - will succeed. - */ - triton_ret_t (*get_serialized_handle)(triton_serialized_handle_type_t handle_type, - void* pointer_to_in_handle, - char* serialized_handle); - - /* - Set an serialized info and get a handle to it. - COMMENTS: - 1. See comment 1 of get_serialized_handle above - - 2. This function both deserializes and sets an serialized data that is maintained for subsequent - references. Example, a valid serialized_handle should subsequently be produced when - get_serialized_handle is invoked with the same handle. - - 3. A call to this function must be coupled with a call to free_serialized_handle below because - serialized data is potentially created. - - 4. Caution: 2 handles to the same serialized info might be different; - handles might not be treated by the net-module like C pointers or even reference-counted pointers. - E.g. two calls to set_serialized_handle that returned h0 and h1 must be coupled with the 2 calls - free_serialized_handle(...,h0) and free_serialized_handle(..., h1) (in any order). So, the 2 calls - free_serialized_handle(...,h1) and free_serialized_handle(..., h1) might be wrong unless something like - free_serialized_handle(...,h1); h1 = h0; free_serialized_handle(..., h1) happens. - */ - triton_ret_t (*create_handle_from_serialization)(triton_serialized_handle_type_t handle_type, - void* pointer_to_out_handle, - char* serialized_handle); - - /* - Free an serialized info of type handle_type and whose handle is *pointer_to_in_handle. - COMMENTS: - 1. This call will(should) succeed only for handles created through set_serialized_handle. - The function will(should) fail if a handle of the same info type created through a different mechanism - is provided. E.g. This function will(should) fail for any attempt to release the info associated - with a triton_mem_pub_handle_t that was directly created through publish_memory. For such a handle - unpublish_memory is the right way to "free" things. - */ - triton_ret_t (*free_serialized_handle)(triton_serialized_handle_type_t handle_type, - void* pointer_to_in_handle); - - /*dest_bufs is a an array of buffer local to the initator of the operation - iov_items_count is the number of iov_items and also the number of bufs - */ - __blocking triton_ret_t (*get)(triton_msg_method_addr_t from, - triton_mem_pub_handle_t handle, - triton_iov_item_t* iovs, - char** dest_bufs, - int iov_items_count); - - /*src_bufs is a an array of buffer local to the initator of the operation - iov_items_count is the number of iov_items and also the number of bufs - */ - __blocking triton_ret_t (*put)(triton_msg_method_addr_t to, - triton_mem_pub_handle_t handle, - triton_iov_item_t* iovs, - char** src_bufs, - int iov_items_count); - - - __blocking triton_ret_t (*start_epoch)(triton_mem_pub_handle_t in_handle); - - __blocking triton_ret_t (*end_epoch)(triton_mem_pub_handle_t in_handle); - - /*Wait_info is netmodule-specific data that defines the wait completion condition - */ - __blocking triton_ret_t (*wait)(triton_mem_pub_handle_t handle, void* wait_info); -}; - -/** - * Methods must register their implementations to the message layer. - */ -triton_ret_t triton_msg_method_register(struct triton_msg_method *method); - -/** - * Unregister a method if its being finalized. - */ -void triton_msg_method_unregister(const char *name); - -/** - * Lookup a registered method based on method name. - */ -struct triton_msg_method* triton_msg_lookup_method(const char* method); - -#endif diff --git a/code/src/net/triton-message.ae b/code/src/net/triton-message.ae deleted file mode 100644 index 6865ec3..0000000 --- a/code/src/net/triton-message.ae +++ /dev/null @@ -1,666 +0,0 @@ - -#include <triton-base.h> -#include <aesop/aesop.h> -#include "src/net/triton-message.hae" -#include "src/net/triton-message-method.hae" -#include <triton-hash.h> - -static triton_mutex_t module_lock = TRITON_MUTEX_INITIALIZER; -static int module_refcount = 0; - -triton_addr_t triton_addr_null = TRITON_ADDR_NULL_STATIC_INITIALIZER; - -/** - * Methods register themselves through the triton_msg_method_register call. - * Once registered, an entry in the method_table keeps track of a method - * keyed on its name. The entry also contains a list of group structures - * so that the group structures can be freed once the method entry structure - * is removed from the method_table. - */ - -static struct triton_hash_table *method_table = NULL; - -/** - * returned method is NULL if given triton_addr_null - */ -static inline void addr_split( - const triton_addr_t addr, struct triton_msg_method **method, triton_msg_method_addr_t *method_addr) -{ - if(method) *method = (struct triton_msg_method *)addr.u; - if(method_addr) *method_addr = addr.l; -} - -static triton_addr_t addr_wrap(struct triton_msg_method *method, triton_msg_method_addr_t method_addr) -{ - triton_addr_t addr; - addr.u = (uint64_t)(uintptr_t)method; - addr.l = method_addr; - return addr; -} - -struct triton_msg_group -{ - uint32_t group; - struct triton_msg_method *method; - struct triton_list_link link; -}; - -struct method_entry -{ - char *name; - struct triton_msg_method *method; - triton_list_t groups; - struct triton_hash_link link; - struct triton_hash_link id_link; -}; - -static int method_compare(const void *k, struct triton_hash_link *l) -{ - struct method_entry *m = triton_hash_get_entry(l, struct method_entry, link); - return (!strcmp(m->method->name, (const char *)k)); -} - -triton_ret_t triton_msg_init(void) -{ - triton_mutex_lock(&module_lock); - if(!module_refcount) - { - method_table = triton_hash_init(method_compare, triton_hash_strhash, 1024); - if(!method_table) - { - triton_mutex_unlock(&module_lock); - return TRITON_ERR_NOMEM; - } - } - module_refcount++; - triton_mutex_unlock(&module_lock); - return TRITON_SUCCESS; -} - -static void free_method(void *x) -{ - struct triton_msg_group *g1, *g2; - struct method_entry *m = (struct method_entry *)x; - triton_list_for_each_entry(g1, g2, &m->groups, struct triton_msg_group, link) - { - triton_list_del(&g1->link); - free(g1); - } - - free(m->name); - free(m); -} - -void triton_msg_finalize(void) -{ - triton_mutex_lock(&module_lock); - module_refcount--; - - if(!module_refcount) - { - triton_hash_destroy_and_finalize(method_table, struct method_entry, link, free_method); - method_table = NULL; - } - triton_mutex_unlock(&module_lock); -} - -triton_ret_t triton_msg_method_register(struct triton_msg_method *method) -{ - struct method_entry *m; - triton_ret_t ret; - struct triton_hash_link *l; - - assert(method != NULL); - - ret = triton_msg_init(); - triton_error_assert(ret); - - l = triton_hash_search(method_table, &method->name); - if(l) - { - /* method with this name already exists! Either the same method is being - * registered, or another method is trying to use an method already in use. - */ - return TRITON_ERR_METHOD_ID_INUSE; - } - - m = malloc(sizeof(*m)); - assert(m); - - m->method = method; - m->name = strdup(m->method->name); - triton_list_init(&m->groups); - - triton_list_link_clear(&m->link); - triton_hash_add(method_table, m->name, &m->link); - - return TRITON_SUCCESS; -} - -void triton_msg_method_unregister(const char *name) -{ - struct triton_hash_link *link; - struct method_entry *m; - - link = triton_hash_search_and_remove(method_table, name); - m = triton_hash_get_entry(link, struct method_entry, link); - free(m->name); - free(m); -} - -/* finds the method structure matching the specified name */ -struct triton_msg_method* triton_msg_lookup_method(const char* method) -{ - struct triton_hash_link *link; - struct method_entry *m; - - link = triton_hash_search(method_table, method); - if(!link) return NULL; - m = triton_hash_get_entry(link, struct method_entry, link); - return(m->method); -} - -triton_ret_t triton_msg_group_add(const char *method, const char *service_name, triton_msg_group_t *group) -{ - triton_ret_t ret; - struct triton_hash_link *link; - struct triton_msg_group *g; - struct method_entry *m; - - link = triton_hash_search(method_table, method); - if(!link) return TRITON_ERR_MSG_METHOD_NOTFOUND; - m = triton_hash_get_entry(link, struct method_entry, link); - - g = malloc(sizeof(*g)); - if(!g) - { - return TRITON_ERR_NOMEM; - } - - g->method = m->method; - ret = m->method->group_add(service_name, &g->group); - if(ret != TRITON_SUCCESS) - { - free(g); - return ret; - } - - /* need to keep track of groups so we can free stuff when a method is disabled */ - triton_list_link_clear(&g->link); - triton_queue_enqueue(&g->link, &m->groups); - - *group = g; - return TRITON_SUCCESS; -} - -triton_msg_tag_t triton_msg_new_tag(triton_msg_group_t group) -{ - struct triton_msg_group *g = (struct triton_msg_group *)group; - return g->method->new_tag(g->group); -} - -__blocking triton_ret_t triton_msg_send( - triton_addr_t to, - triton_msg_group_t group, - triton_msg_tag_t tag, - int count, - char **buffers, - uint32_t *sizes) -{ - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - struct triton_msg_group *g = (struct triton_msg_group *)group; - - addr_split(to, &tm, &maddr); - assert(tm); - - /* group method should be the same as the to method. - * TODO: Make this a real error. - */ - assert(g->method == tm); - - return g->method->send(maddr, g->group, tag, count, buffers, sizes); -} - -__blocking triton_ret_t triton_msg_recv( - triton_addr_t from, - triton_msg_group_t group, - triton_msg_tag_t tag, - int count, - char **buffers, - uint32_t *sizes, - uint32_t *bytes_received) -{ - struct triton_msg_method *fm; - triton_msg_method_addr_t maddr; - struct triton_msg_group *g = (struct triton_msg_group *)group; - - addr_split(from, &fm, &maddr); - assert(fm); - - /* from method should match group method - * TODO: fix this to be a real error */ - assert(fm == g->method); - - return g->method->recv(maddr, g->group, tag, count, buffers, sizes, bytes_received); -} - -__blocking triton_ret_t triton_msg_recv_any( - triton_msg_group_t group, - triton_addr_t *from, - triton_msg_tag_t *tag, - int count, - char **buffers, - uint32_t *sizes, - uint32_t *bytes_received) -{ - triton_ret_t ret; - triton_msg_method_addr_t method_from; - - struct triton_msg_group *g = (struct triton_msg_group *)group; - ret = g->method->recv_any(g->group, &method_from, tag, count, buffers, sizes, bytes_received); - if(ret != TRITON_SUCCESS) - { - return ret; - } - - /* set the from field properly */ - *from = addr_wrap(g->method, method_from); - return TRITON_SUCCESS; -} - -/** - * Address functions. - */ -int triton_addr_equal(const triton_addr_t a1, const triton_addr_t a2) -{ - struct triton_msg_method *m1, *m2; - triton_msg_method_addr_t ma1, ma2; - - addr_split(a1, &m1, &ma1); - assert(m1); - addr_split(a2, &m2, &ma2); - assert(m2); - return (m1 == m2) && m1->addr_equal(ma1, ma2); -} - -int triton_addr_is_null(const triton_addr_t a) -{ - if(a.u == 0 && a.l == 0) - return 1; - else - return 0; -} - -__blocking int triton_addr_is_self(triton_addr_t addr) -{ - int status; - struct triton_msg_method *method = NULL; - triton_msg_method_addr_t method_addr, self; - addr_split(addr, &method, &method_addr); - triton_assert(method); - self = method->addr_self(); - status = method->addr_equal(method_addr, self); - method->addr_free(self); - return status; -} - -__blocking triton_addr_t triton_addr_self(const char *method) -{ - triton_msg_method_addr_t maddr; - triton_ret_t ret; - - struct triton_msg_method *m = triton_msg_lookup_method(method); - if(!m) - { - return(triton_addr_null); - } - maddr = m->addr_self(); - return addr_wrap(m, maddr); -} - -triton_ret_t triton_addr_free(triton_addr_t addr) -{ - struct triton_msg_method *method = NULL; - triton_msg_method_addr_t method_addr; - - /* TODO: need a utility function/macro for this comparison */ - if(addr.u == 0 && addr.l == 0) - { - /* NULL address, nothing to do */ - return TRITON_SUCCESS; - } - - addr_split(addr, &method, &method_addr); - triton_assert(method); - return method->addr_free(method_addr); -} - -triton_ret_t triton_addr_copy(const triton_addr_t orig_addr, triton_addr_t *copy) -{ - struct triton_msg_method *method = NULL; - triton_msg_method_addr_t orig_method_addr, copy_method_addr; - addr_split(orig_addr, &method, &orig_method_addr); - triton_assert(method); - method->addr_copy(orig_method_addr, &copy_method_addr); - *copy = addr_wrap(method, copy_method_addr); - return TRITON_SUCCESS; -} - - -triton_ret_t triton_addr_to_string(triton_addr_t addr, triton_string_t *str) -{ - triton_ret_t ret; - triton_string_t methodstr; - struct triton_msg_method *m; - triton_msg_method_addr_t maddr; - - addr_split(addr, &m, &maddr); - if(m == NULL) - { - return TRITON_ERR_INVAL; - } - - ret = m->addr_to_string(maddr, &methodstr); - if(ret == TRITON_SUCCESS) - { - triton_string_init(str, "%s://%s", m->name, methodstr.string); - } - triton_string_destroy(&methodstr); - return ret; -} - -triton_ret_t triton_addr_lookup(const char *name, triton_addr_t *addr) -{ - char *n; - char prefix[128]; - char method_addr[1024]; - triton_msg_method_addr_t maddr; - triton_ret_t ret; - - /* parse addr endpoint prefix, so that we can lookup the method */ - - n = index(name, ':'); - if(n[1] != '/' || n[2] != '/') - { - /* TODO: return a proper error here */ - return TRITON_ERR_INVAL; - } - strncpy(prefix, name, (n - name)); - prefix[n-name] = '\0'; - - n += 3; - strcpy(method_addr, n); - - struct triton_msg_method *m = triton_msg_lookup_method(prefix); - if(!m) - { - /* TODO: Add some debugging and/or wrap the error up with prefix and method_addr info */ - return TRITON_ERR_MSG_METHOD_NOTFOUND; - } - - ret = m->addr_lookup(method_addr, &maddr); - if(ret != TRITON_SUCCESS) - { - return ret; - } - - *addr = addr_wrap(m, maddr); - return TRITON_SUCCESS; -} - - -/** -*Set the message size beyond which one-sided communication is used instead 2-sided for message transfers. -* The size is in bytes -*/ -triton_ret_t triton_set_one_sided_threshold(triton_addr_t local_addr, size_t size) -{ - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - addr_split(local_addr, &tm, &maddr); - return tm->set_one_sided_threshold(size); -} - - -/** -*Get the message size beyond which one-sided communication is used instead 2-sided for message transfers. -* The size is in bytes -*/ -triton_ret_t triton_get_one_sided_threshold(triton_addr_t local_addr, size_t *size) -{ - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - addr_split(local_addr, &tm, &maddr); - return tm->get_one_sided_threshold(size); -} - -/** - * Allocate a buffer for sending and receiving. Method can use this - * call to pin a buffer. - * TODO: Should we include information about the type of buffer to allocate? - */ -void* triton_buffer_allocate(triton_addr_t local_addr, size_t size) -{ - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - addr_split(local_addr, &tm, &maddr); - return tm->buffer_allocate(size); -} - -/** - * Free a buffer allocated with buffer_allocate. - */ -void triton_buffer_free(triton_addr_t local_addr, void *buffer) -{ - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - addr_split(local_addr, &tm, &maddr); - tm->buffer_free(buffer); -} - -/** -* Register memory for the communication subsystem. -*/ -triton_ret_t triton_register_memory(triton_addr_t local_addr, void* base, - size_t size, - triton_mem_reg_handle_t* handle) -{ - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - addr_split(local_addr, &tm, &maddr); - return tm->register_memory(base, size, handle); -} -/** -* Unregister memory. -*/ -triton_ret_t triton_unregister_memory(triton_addr_t local_addr, triton_mem_reg_handle_t handle) -{ - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - addr_split(local_addr, &tm, &maddr); - return tm->unregister_memory(handle); -} - -/** -* Expose a registered memory portion to remote peers. -*/ -__blocking triton_ret_t triton_publish_memory(triton_addr_t local_addr, - triton_mem_reg_handle_t in_handle, - size_t offset, - size_t size, - triton_mem_pub_handle_t* out_handle) -{ - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - addr_split(local_addr, &tm, &maddr); - return tm->publish_memory(in_handle, offset, size, out_handle); -} - -/** -* End access permissions to a memory portion previously exposed. -*/ -__blocking triton_ret_t triton_unpublish_memory(triton_addr_t local_addr, triton_mem_pub_handle_t handle) -{ - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - addr_split(local_addr, &tm, &maddr); - return tm->unpublish_memory(handle); -} - - - - /*returns the size of buffer required to get the serialized representation - of the info type passed as argument - */ -int triton_get_serialized_handle_size(triton_addr_t local_addr, - triton_serialized_handle_type_t handle_type, void* pointer_to_handle) -{ - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - addr_split(local_addr, &tm, &maddr); - return tm->get_serialized_handle_size(handle_type, pointer_to_handle); -} - - -/*Ask the net-module to provide the serialized net-module-specific info - associated with the handle *pointer_to_in_handle. - COMMENTS: - 1. pointer_to_in_handle must point to a handle compatible with handle_type - (e.g. if handle_type is MEM_PUB_HANDLE_TYPE, then pointer_to_in_handle must - point to a triton_mem_pub_handle_t). - - 2. serialized_handle must be large enough. A previous call to - get_serialized_handle_size is recommended to determine the required buffer size. - - 3. This function might succeed even if triton_create_handle_from_serialization (below) was not called previously - to return pointer_to_in_handle. E.g. When publish_memory returns a handle h, any - subsequent call to get_serialized_handle with h and handle_type == MEM_PUB_HANDLE_TYPE - will succeed. -*/ -triton_ret_t triton_get_serialized_handle(triton_addr_t local_addr, - triton_serialized_handle_type_t handle_type, - void* pointer_to_in_handle, - char* serialized_handle) -{ - - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - addr_split(local_addr, &tm, &maddr); - return tm->get_serialized_handle(handle_type, pointer_to_in_handle, serialized_handle); -} - -/* - Create a handle from serialized info and get a handle to it. - COMMENTS: - 1. See comment 1 of get_serialized_handle above - - 2. This function both deserializes and sets an serialized data that is maintained for subsequent - references. Example, a valid serialized_handle should subsequently be produced when - get_serialized_handle is invoked with the same handle. - - 3. A call to this function must be coupled with a call to free_serialized_handle below because - serialized data is potentially created. - - 4. Caution: 2 handles to the same serialized info might be different; - handles might not be treated by the net-module like C pointers or even reference-counted pointers. - E.g. two calls to set_serialized_handle that returned h0 and h1 must be coupled with the 2 calls - free_serialized_handle(...,h0) and free_serialized_handle(..., h1) (in any order). So, the 2 calls - free_serialized_handle(...,h1) and free_serialized_handle(..., h1) might be wrong unless something like - free_serialized_handle(...,h1); h1 = h0; free_serialized_handle(..., h1) happens. -*/ -triton_ret_t triton_create_handle_from_serialization(triton_addr_t local_addr, - triton_serialized_handle_type_t handle_type, - void* pointer_to_out_handle, - char* serialized_handle) -{ - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - addr_split(local_addr, &tm, &maddr); - return tm->create_handle_from_serialization(handle_type, pointer_to_out_handle, serialized_handle); -} - -/* - Free an serialized info of type handle_type and whose handle is *pointer_to_in_handle. - COMMENTS: - 1. This call will(should) succeed only for handles created through set_serialized_handle. - The function will(should) fail if a handle of the same info type created through a different mechanism - is provided. E.g. This function will(should) fail for any attempt to release the info associated - with a triton_mem_pub_handle_t that was directly created through publish_memory. For such a handle - unpublish_memory is the right way to "free" things. -*/ -triton_ret_t triton_free_serialized_handle(triton_addr_t local_addr, triton_serialized_handle_type_t handle_type, - void* pointer_to_in_handle) -{ - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - addr_split(local_addr, &tm, &maddr); - return tm->free_serialized_handle(handle_type, pointer_to_in_handle); -} - -/*dest_bufs is a an array of buffer local to the initator of the operation -iov_items_count is the number of iov_items and also the number of bufs -*/ -__blocking triton_ret_t triton_msg_get(triton_addr_t local_addr, - triton_addr_t from, - triton_mem_pub_handle_t handle, - triton_iov_item_t* iovs, - char **dest_bufs, - int iov_items_count) -{ - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - addr_split(from, &tm, &maddr); - return tm->get(maddr, handle, iovs, dest_bufs, iov_items_count); -} - -/*src_bufs is a an array of buffer local to the initator of the operation -iov_items_count is the number of iov_items and also the number of bufs -*/ -__blocking triton_ret_t triton_msg_put(triton_addr_t local_addr, - triton_addr_t to, - triton_mem_pub_handle_t handle, - triton_iov_item_t* iovs, - char ** src_bufs, - int iov_items_count) -{ - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - addr_split(to, &tm, &maddr); - return tm->put(maddr, handle, iovs, src_bufs, iov_items_count); -} - - -__blocking triton_ret_t triton_start_epoch(triton_addr_t local_addr, triton_mem_pub_handle_t in_handle) -{ - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - addr_split(local_addr, &tm, &maddr); - return tm->start_epoch(in_handle); -} - -__blocking triton_ret_t triton_end_epoch(triton_addr_t local_addr, triton_mem_pub_handle_t in_handle) -{ - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - addr_split(local_addr, &tm, &maddr); - return tm->end_epoch(in_handle); -} - -__blocking triton_ret_t triton_wait(triton_addr_t local_addr, triton_mem_pub_handle_t handle, - void* wait_info) -{ - struct triton_msg_method *tm; - triton_msg_method_addr_t maddr; - addr_split(local_addr, &tm, &maddr); - return tm->wait(handle, wait_info); -} - -/* - * Local variables: - * c-indent-level: 4 - * c-basic-offset: 4 - * End: - * - * vim: ts=8 sts=4 sw=4 expandtab - */ diff --git a/code/src/net/triton-message.hae b/code/src/net/triton-message.hae deleted file mode 100644 index 3eb0570..0000000 --- a/code/src/net/triton-message.hae +++ /dev/null @@ -1,269 +0,0 @@ -#ifndef __TRITON_MESSAGE_HAE__ -#define __TRITON_MESSAGE_HAE__ - -#include <triton-string.h> -#include "src/common/triton-types.h" -#include "src/common/triton-error.h" -#include <aesop/aesop.h> -#include "src/net/triton-addr.h" - -/** - * functions to do address lookup and comparison. - */ -__blocking triton_addr_t triton_addr_self(const char *method); - -__blocking int triton_addr_is_self(triton_addr_t addr); - -triton_ret_t triton_addr_free(triton_addr_t addr); - -triton_ret_t triton_addr_copy(const triton_addr_t orig_addr, triton_addr_t *copy); -/** - * Right now we use 32-bits for the tag. - */ -typedef int32_t triton_msg_tag_t; - -/** - * Groups allow users of the message interface to logically group messages together - * based on a particular service. This provides scoping for the message tag values - * (tags from different groups can have the same value) and also allows different - * types of messages to be received. For example, unexpected messages could be received - * using recv_any on an unexpected group, while expected messages can be received - * using the recv call with a tag specified. - * - * At the moment, the MPI method simply creates a new communicator for each new group - * that gets added. Various interfaces could create separate groups and send messages - * using different groups. - */ - -struct triton_msg_group; -typedef struct triton_msg_group *triton_msg_group_t; - -/** - * Initialize the messaging interface. - */ -triton_ret_t triton_msg_init(void); - -/** - * Finalize the messaging interface. - */ -void triton_msg_finalize(void); - -/** - * Add a message group. Examples: - * - * Allow for unexpected messages to be received via recv_any, without also receiving expected receives. - * - * group_add(method_name, "aesop.remote.unexpected", &ae_remote_unexp_group); - * group_add(method_name, "aesop.remote.expexted", &ae_remote_exp_group); - */ -triton_ret_t triton_msg_group_add(const char *method_name, const char *service_name, triton_msg_group_t *group); - -/** - * Get a new tag from the message context. - */ -triton_msg_tag_t triton_msg_new_tag(triton_msg_group_t group); - -/** - * Sends a message using the method the group was created on. - * Group and tag fields must be specified to allow for proper matching at the receive side. - */ -__blocking triton_ret_t triton_msg_send( - triton_addr_t to, - triton_msg_group_t group, - triton_msg_tag_t tag, - int count, - char **buffers, - uint32_t *sizes); - -/** - * Receive a message using the method the group was created on. - * The received message will match on the group and tag fields passed in - * and the message buffers will be filled. This call does an expected receive. - */ -__blocking triton_ret_t triton_msg_recv( - triton_addr_t from, - triton_msg_group_t group, - triton_msg_tag_t tag, - int count, - char **buffers, - uint32_t *sizes, - uint32_t *bytes_received); - -__blocking triton_ret_t triton_msg_recv_any( - triton_msg_group_t group, - triton_addr_t *from, - triton_msg_tag_t *tag, - int count, - char **buffers, - uint32_t *sizes, - uint32_t *bytes_received); - - -typedef uint64_t triton_mem_reg_handle_t; - -typedef uint64_t triton_mem_pub_handle_t; - -typedef struct triton_iov_item_t -{ - size_t offset; - size_t size; -}triton_iov_item_t; -/* -See get_serialized_handle below -*/ -typedef enum triton_serialized_handle_type_t -{ - MEM_PUB_HANDLE_TYPE, -}triton_serialized_handle_type_t; - - -/** -*Set the message size beyond which one-sided communication is used instead 2-sided for message transfers. -* The size is in bytes -*/ -triton_ret_t triton_set_one_sided_threshold(triton_addr_t local_addr, size_t size); - - -/** -*Get the message size beyond which one-sided communication is used instead 2-sided for message transfers. -* The size is in bytes -*/ -triton_ret_t triton_get_one_sided_threshold(triton_addr_t local_addr, size_t *size); - -/** - * Allocate a buffer for sending and receiving. Method can use this - * call to pin a buffer. - * TODO: Should we include information about the type of buffer to allocate? - */ -void * triton_buffer_allocate(triton_addr_t local_addr, size_t size); - -/** - * Free a buffer allocated with buffer_allocate. - */ -void triton_buffer_free(triton_addr_t local_addr, void *buffer); - -/** -* Register memory for the communication subsystem. -*/ -triton_ret_t triton_register_memory(triton_addr_t local_addr, void* base, - size_t size, - triton_mem_reg_handle_t* handle); -/** -* Unregister memory. -*/ -triton_ret_t triton_unregister_memory(triton_addr_t local_addr, triton_mem_reg_handle_t handle); - -/** -* Expose a registered memory portion to remote peers. -*/ -__blocking triton_ret_t triton_publish_memory(triton_addr_t local_addr, triton_mem_reg_handle_t in_handle, - size_t offset, - size_t size, - triton_mem_pub_handle_t* out_handle); - -/** -* End access permissions to a memory portion previously exposed. -*/ -__blocking triton_ret_t triton_unpublish_memory(triton_addr_t local_addr, triton_mem_pub_handle_t handle); - - - /*returns the size of buffer required to get the serialized representation - of the info type passed as argument - */ -int triton_get_serialized_handle_size(triton_addr_t local_addr, triton_serialized_handle_type_t handle_type, - void* pointer_to_handle); - - -/*Ask the net-module to provide the serialized net-module-specific info - associated with the handle *pointer_to_in_handle. - COMMENTS: - 1. pointer_to_in_handle must point to a handle compatible with handle_type - (triton_addr_t local_addr, e.g. if handle_type is MEM_PUB_HANDLE_TYPE, then pointer_to_in_handle must - point to a triton_mem_pub_handle_t). - - 2. serialized_handle must be large enough. A previous call to - get_serialized_handle_size is recommended to determine the required buffer size. - - 3. This function might succeed even if set_serialized_handle (triton_addr_t local_addr, below) was not called previously - to return pointer_to_in_handle. E.g. When publish_memory returns a handle h, any - subsequent call to get_serialized_handle with h and handle_type == MEM_PUB_HANDLE_TYPE - will succeed. -*/ -triton_ret_t triton_get_serialized_handle(triton_addr_t local_addr, triton_serialized_handle_type_t handle_type, - void* pointer_to_in_handle, - char* serialized_handle); - -/* - Create an serialized info and get a handle to it. - COMMENTS: - 1. See comment 1 of get_serialized_handle above - - 2. This function both deserializes and sets an serialized data that is maintained for subsequent - references. Example, a valid serialized_handle should subsequently be produced when - get_serialized_handle is invoked with the same handle. - - 3. A call to this function must be coupled with a call to free_serialized_handle below because - serialized data is potentially created. - - 4. Caution: 2 handles to the same serialized info might be different; - handles might not be treated by the net-module like C pointers or even reference-counted pointers. - E.g. two calls to set_serialized_handle that returned h0 and h1 must be coupled with the 2 calls - free_serialized_handle(triton_addr_t local_addr, ...,h0) and free_serialized_handle(triton_addr_t local_addr, ..., h1) (triton_addr_t local_addr, in any order). So, the 2 calls - free_serialized_handle(triton_addr_t local_addr, ...,h1) and free_serialized_handle(triton_addr_t local_addr, ..., h1) might be wrong unless something like - free_serialized_handle(triton_addr_t local_addr, ...,h1); h1 = h0; free_serialized_handle(triton_addr_t local_addr, ..., h1) happens. -*/ -triton_ret_t triton_create_handle_from_serialization(triton_addr_t local_addr, triton_serialized_handle_type_t handle_type, - void* pointer_to_out_handle, - char* serialized_handle); - -/* - Free an serialized info of type handle_type and whose handle is *pointer_to_in_handle. - COMMENTS: - 1. This call will(triton_addr_t local_addr, should) succeed only for handles created through set_serialized_handle. - The function will(triton_addr_t local_addr, should) fail if a handle of the same info type created through a different mechanism - is provided. E.g. This function will(triton_addr_t local_addr, should) fail for any attempt to release the info associated - with a triton_mem_pub_handle_t that was directly created through publish_memory. For such a handle - unpublish_memory is the right way to "free" things. -*/ -triton_ret_t triton_free_serialized_handle(triton_addr_t local_addr, triton_serialized_handle_type_t handle_type, - void* pointer_to_in_handle); - -/*dest_bufs is a an array of buffer local to the initator of the operation -iov_items_count is the number of iov_items and also the number of bufs -*/ -__blocking triton_ret_t triton_msg_get(triton_addr_t local_addr, triton_addr_t from, - triton_mem_pub_handle_t handle, - triton_iov_item_t* iovs, - char** dest_bufs, - int iov_items_count); - -/*src_bufs is a an array of buffer local to the initator of the operation -iov_items_count is the number of iov_items and also the number of bufs -*/ -__blocking triton_ret_t triton_msg_put(triton_addr_t local_addr, triton_addr_t to, - triton_mem_pub_handle_t handle, - triton_iov_item_t* iovs, - char** src_bufs, - int iov_items_count); - - -__blocking triton_ret_t triton_start_epoch(triton_addr_t local_addr, triton_mem_pub_handle_t in_handle); - -__blocking triton_ret_t triton_end_epoch(triton_addr_t local_addr, triton_mem_pub_handle_t in_handle); - -/*Wait_info is netmodule-specific data that defines the wait completion condition -*/ -__blocking triton_ret_t triton_wait(triton_addr_t local_addr, triton_mem_pub_handle_t handle, - void* wait_info); - - -#endif /* __TRITON_MESSAGE_HAE__ */ - -/* - * Local variables: - * c-indent-level: 4 - * c-basic-offset: 4 - * End: - * - * vim: ts=8 sts=4 sw=4 expandtab - */ diff --git a/code/src/remote/module.mk.in b/code/src/remote/module.mk.in index fa3e751..79b389b 100644 --- a/code/src/remote/module.mk.in +++ b/code/src/remote/module.mk.in @@ -1,16 +1,17 @@ DIR_REMOTE := src/remote -AESOP_HDR += $(DIR_REMOTE)/message.hae \ - $(DIR_REMOTE)/service.hae \ +AESOP_HDR += \ $(DIR_REMOTE)/message-method.hae \ $(DIR_REMOTE)/remote.hae +# $(DIR_REMOTE)/message.hae \ +# $(DIR_REMOTE)/service.hae \ # $(DIR_REMOTE)/client-server.hae -- Gone? -Justin -AELIBSRC += $(DIR_REMOTE)/message.ae \ - $(DIR_REMOTE)/service.ae \ - $(DIR_REMOTE)/retry-method.ae - +#AELIBSRC += \ +# $(DIR_REMOTE)/retry-method.ae +# $(DIR_REMOTE)/service.ae \ +# $(DIR_REMOTE)/message.ae \ # $(DIR_REMOTE)/core-client.ae # $(DIR_REMOTE)/client-server-exec.ae \ hooks/post-receive --

1 0

branch, devel-mercury, updated. 229351495c9429cd8bd3db518478da4e485ec17e
by noreply＠mcs.anl.gov 31 Jul '13

31 Jul '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, devel-mercury has been updated via 229351495c9429cd8bd3db518478da4e485ec17e (commit) via 59756e5fe9f863f5338556fa537a17269573e964 (commit) via 542850c5ce533c511996e93752256f2986922ff9 (commit) via 3949430cda6829e0cc61d05b060ee6f647e8f783 (commit) from 6570093903f4ddbf16cf2f71c266b46b245dbb0b (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 229351495c9429cd8bd3db518478da4e485ec17e Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jul 31 17:17:06 2013 -0400 remove aercc parser and disable aer compilation commit 59756e5fe9f863f5338556fa537a17269573e964 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jul 31 16:47:36 2013 -0400 bug fixes in aesop and mercury checks commit 542850c5ce533c511996e93752256f2986922ff9 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jul 31 16:25:01 2013 -0400 make aesop and mercury hard dependencies commit 3949430cda6829e0cc61d05b060ee6f647e8f783 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jul 31 15:34:42 2013 -0400 remove deprecated configure tests ----------------------------------------------------------------------- Summary of changes: code/Makefile.in | 148 +--- code/configure.ac | 116 +-- code/maint/aercc.in | 211 ---- code/maint/config/ghc.m4 | 26 - code/maint/depend-aer.sh | 23 - code/maint/hs/.gitignore | 9 - code/maint/hs/README.txt | 2 - code/maint/hs/language-c-0.4.2.patch | 309 ------ code/maint/hs/language-c-0.4.tar.gz | Bin 108164 -> 0 bytes code/maint/hs/language-c.patch | 317 ------ code/maint/hs/setup-aesop | 47 - code/maint/hs/setup-cabal-local | 11 - code/maint/hs/setup-hs-local | 12 - code/src/aesop/ae-remote-parser.h | 566 ---------- code/src/aesop/parser/BlockingContext.lhs | 65 -- code/src/aesop/parser/CGen.lhs | 787 -------------- code/src/aesop/parser/CParse.lhs | 470 --------- code/src/aesop/parser/CUtils.lhs | 114 -- code/src/aesop/parser/ContextGen.lhs | 53 - code/src/aesop/parser/Header.lhs | 99 -- code/src/aesop/parser/Serialize.hs | 139 --- code/src/aesop/parser/Walker.lhs | 352 ------- code/src/aesop/parser/ae-remote-parser.lhs | 1104 -------------------- code/src/aesop/parser/ae-test.lhs | 38 - code/src/aesop/parser/module.mk.in | 14 - .../aesop/parser/tests/blocking/parallelfor4.ae | 93 -- code/src/aesop/parser/tests/remote/module.mk.in | 16 - .../tests/remote/test-remote-buffer-response.aer | 110 -- .../parser/tests/remote/test-remote-error-wrap.aer | 121 --- .../tests/remote/test-remote-fault-injector.aer | 231 ---- .../aesop/parser/tests/remote/test-remote-list.aer | 262 ----- .../aesop/parser/tests/remote/test-remote-list.sh | 14 - .../aesop/parser/tests/remote/test-remote-mpi.aer | 181 ---- .../tests/remote/test-remote-service-stop.aer | 113 -- .../aesop/parser/tests/syntax/err-untyped-func.ae | 18 - .../parser/tests/syntax/err-untyped-func.errs | 1 - code/src/aesop/parser/tests/syntax/run-tests.sh | 48 - code/src/rebuild/module.mk.in | 12 +- code/src/remote/module.mk.in | 9 +- code/src/replicated-osd/module.mk.in | 20 +- code/src/state/module.mk.in | 11 +- 41 files changed, 45 insertions(+), 6247 deletions(-) delete mode 100644 code/maint/aercc.in delete mode 100644 code/maint/config/ghc.m4 delete mode 100755 code/maint/depend-aer.sh delete mode 100644 code/maint/hs/.gitignore delete mode 100644 code/maint/hs/README.txt delete mode 100644 code/maint/hs/language-c-0.4.2.patch delete mode 100644 code/maint/hs/language-c-0.4.tar.gz delete mode 100644 code/maint/hs/language-c.patch delete mode 100755 code/maint/hs/setup-aesop delete mode 100755 code/maint/hs/setup-cabal-local delete mode 100755 code/maint/hs/setup-hs-local delete mode 100644 code/src/aesop/ae-remote-parser.h delete mode 100644 code/src/aesop/parser/BlockingContext.lhs delete mode 100644 code/src/aesop/parser/CGen.lhs delete mode 100644 code/src/aesop/parser/CParse.lhs delete mode 100644 code/src/aesop/parser/CUtils.lhs delete mode 100644 code/src/aesop/parser/ContextGen.lhs delete mode 100644 code/src/aesop/parser/Header.lhs delete mode 100644 code/src/aesop/parser/Serialize.hs delete mode 100644 code/src/aesop/parser/Walker.lhs delete mode 100644 code/src/aesop/parser/ae-remote-parser.lhs delete mode 100644 code/src/aesop/parser/ae-test.lhs delete mode 100644 code/src/aesop/parser/module.mk.in delete mode 100644 code/src/aesop/parser/tests/blocking/parallelfor4.ae delete mode 100644 code/src/aesop/parser/tests/remote/module.mk.in delete mode 100644 code/src/aesop/parser/tests/remote/test-remote-buffer-response.aer delete mode 100644 code/src/aesop/parser/tests/remote/test-remote-error-wrap.aer delete mode 100644 code/src/aesop/parser/tests/remote/test-remote-fault-injector.aer delete mode 100644 code/src/aesop/parser/tests/remote/test-remote-list.aer delete mode 100755 code/src/aesop/parser/tests/remote/test-remote-list.sh delete mode 100644 code/src/aesop/parser/tests/remote/test-remote-mpi.aer delete mode 100644 code/src/aesop/parser/tests/remote/test-remote-service-stop.aer delete mode 100644 code/src/aesop/parser/tests/syntax/err-untyped-func.ae delete mode 100644 code/src/aesop/parser/tests/syntax/err-untyped-func.errs delete mode 100755 code/src/aesop/parser/tests/syntax/run-tests.sh Diff of changes: diff --git a/code/Makefile.in b/code/Makefile.in index 4f52958..122f91d 100644 --- a/code/Makefile.in +++ b/code/Makefile.in @@ -56,9 +56,6 @@ ifneq (,@AE_DEBUG@) AEDEBUG = @AE_DEBUG@ endif -# for now assume ghc is installed and in path. -GHC = @GHC@ - CC = @CC@ AESOP = @USE_AESOP@ @@ -100,7 +97,7 @@ CWARNS += -Wno-format-security # also don't warn if we user zero-length formats in printf-style functions CWARNS += -Wno-format-zero-length -LDSHARED = $(TCC) -shared +LDSHARED = $(CC) -shared LIBCFLAGS=@LIBCFLAGS@ # tell libev to use our config.h instead of its own; we can't just put this @@ -116,34 +113,19 @@ DEPLIBS += @THREAD_LIB@ CPPDEFS=$(filter -D%,$(LIBCFLAGS) $(CFLAGS)) CPPINCS=$(filter -I%,$(LIBCFLAGS) $(CFLAGS)) -MPICC = @MPICC@ -MPICFLAGS = @MPICFLAGS@ -MPILDFLAGS = @MPILDFLAGS@ -MPILIBS = @MPILIBS@ -BUILD_MPI = @BUILD_MPI@ - USE_MERCURY=@USE_MERCURY@ -CFLAGS += $(MPICFLAGS) -LDFLAGS += $(MPILDFLAGS) -LIBS += $(MPILIBS) - USE_DYLIBS = @USE_DYLIBS@ INSTALL = @INSTALL@ AELIBSRC := -AERLIBSRC := ASGLIBSRC := LIBSRC := AETESTSRC := -AERTESTSRC := AETESTSUPPORTSRC := AEPROGRAMSRC := -AERPROGRAMSRC := AESOP_HDR := -HSSRC := -HSBINSRC := TESTSRC := TESTSUPPORTSRC := PROGRAMSRC := @@ -209,7 +191,7 @@ endif # of a file that has the same name as the target. Listing them # here keeps make from accidentally doing too much work (see GNU # make manual). -.PHONY: all clean parserclean test_results +.PHONY: all clean test_results ################################################################ # Find project subdirectories @@ -302,21 +284,18 @@ canondirname = $(subst /,_,$(1)) modcflags = $(MODCFLAGS_$(call canonname,$(1))) \ $(MODCFLAGS_$(patsubst $(srcdir)/%,%,$(1))) -MODCFLAGS_AER = $(call modcflags,$(1)) MODCFLAGS_AE = $(call modcflags,$(1)) MODCFLAGS_C = $(call modcflags,$(1)) modldflags = $(MODLDFLAGS_$(call canonname,$(1))) \ $(MODLDFLAGS_$(patsubst $(srcdir)/%,%,$(1))) -MODLDFLAGS_AER = $(call modldflags,$(1)) MODLDFLAGS_AE = $(call modldflags,$(1)) MODLDFLAGS_C = $(call modldflags,$(1)) modlibs = $(MODLIBS_$(call canonname,$(1))) \ $(MODLIBS_$(patsubst $(srcdir)/%,%,$(1))) -MODLIBS_AER = $(call modlibs,$(1)) MODLIBS_AE = $(call modlibs,$(1)) MODLIBS_C = $(call modlibs,$(1)) @@ -328,23 +307,16 @@ modname = $(MODNAME_$(call canonname,$(1))) \ # such as main() modlinkwith = $(MODLINKWITH_$(call canonname,$(1)))$(MODLINKWITH_$(patsubst $(srcdir)/%,%,$(1))) -MODLINKWITH_AER = $(call modlinkwith,$(1)) MODLINKWITH_AE = $(call modlinkwith,$(1)) MODLINKWITH_C = $(call modlinkwith,$(1)) -# use TCC variable so that we can keep CC as the system compiler -TCC = $(if $(MPICC),$(MPICC),$(CC)) CPPCC = $(CC) -modcc = $(if $(MODCC_$(1)),$(MODCC_$(1)),$(TCC)) +modcc = $(if $(MODCC_$(1)),$(MODCC_$(1)),$(CC)) -MCC_AER = $(call modcc,$(1)) MCC_AE = $(call modcc,$(1)) MCC_C = $(call modcc,$(1)) -RMOD_REGDIR = $(call dirname,$(1)) -RMOD_LIB_FILES = $(REMOTE_MODLIB_OBJS_$(call canonfname,$(1))) - # Get the parameters for a given test testparams = $(TESTPARAMS_$(1)) @@ -352,10 +324,6 @@ testparams = $(TESTPARAMS_$(1)) AELIBOBJS=$(patsubst %.ae,%.o,$(AELIBSRC)) aelibobjs: $(AELIBOBJS) -# object files for the triton library generated from .aer files -# (.o and _module.o) -AERLIBOBJS=$(patsubst %.aer,%.o,$(AERLIBSRC)) $(patsubst %.aer,%_module.o,$(AERLIBSRC)) - # The object files for the triton library generated from .c files LIBOBJS=$(patsubst %.c,%.o,$(LIBSRC)) LIBDEPS=$(patsubst %.c,%.d,$(LIBSRC)) @@ -373,21 +341,12 @@ PROGRAM=$(patsubst %.c,%,$(PROGRAMSRC)) # The object files for binary tests generated from .ae files AETESTOBJS=$(patsubst %.ae,%.o,$(AETESTSRC)) $(patsubst %.ae,%.o,$(AETESTSUPPORTSRC)) -# object files for binary tests generated from .aer files -AERTESTOBJS=$(patsubst %.aer,%.o,$(AERTESTSRC)) $(patsubst %.aer,%_module.o,$(AERTESTSRC)) - # The binary files from .ae test files AETEST=$(patsubst %.ae,%,$(AETESTSRC)) -# binary file names from .aer test files -AERTEST=$(patsubst %.aer,%,$(AERTESTSRC)) - AEPROGRAMOBJS=$(patsubst %.ae,%.o,$(AEPROGRAMSRC)) AEPROGRAM=$(patsubst %.ae,%,$(AEPROGRAMSRC)) -AERPROGRAMOBJS=$(patsubst %.aer,%.o,$(AERPROGRAMSRC)) -AERPROGRAM=$(patsubst %.aer,%,$(AERPROGRAMSRC)) - # The C header files generated from .hae files AESOP_TRHDR=$(patsubst %.hae,%.h,$(AESOP_HDR)) AESOP_TRHDR_DEPS=$(patsubst %.hae,%.h.d,$(AESOP_HDR)) @@ -395,26 +354,14 @@ AESOP_TRHDR_DEPS=$(patsubst %.hae,%.h.d,$(AESOP_HDR)) # combined list of all .ae objects AEOBJS=$(AETESTOBJS) $(AELIBOBJS) $(AEPROGRAMOBJS) -# combined list of all .aer objects -AEROBJS=$(AERLIBOBJS) $(AERTESTOBJS) $(AERPROGRAMOBJS) - # dependency files for .ae files AEDEPS=$(patsubst %.o,%.ae.d,$(AEOBJS)) -# dependency files for .aer files -AERDEPS=$(patsubst %.aer,%.aer.d,$(AERLIBSRC)) $(patsubst %.aer,%.aer.d,$(AERTESTSRC)) +DEPENDS=$(TESTDEPS) $(LIBDEPS) $(AEDEPS) $(AESOP_TRHDR_DEPS) -HSBIN=$(patsubst %.lhs,%,$(HSBINSRC)) -HSOBJ=$(patsubst %.lhs,%.o,$(filter %.lhs,$(HSSRC))) \ - $(patsubst %.hs,%.o,$(filter %.hs,$(HSSRC))) -HSINT=$(patsubst %.lhs,%.hi,$(filter %.lhs,$(HSSRC))) \ - $(patsubst %.hs,%.hi,$(filter %.hs,$(HSSRC))) +PROGRAMS = $(AEPROGRAM) $(PROGRAM) -DEPENDS=$(TESTDEPS) $(LIBDEPS) $(AERDEPS) $(AEDEPS) $(AESOP_TRHDR_DEPS) - -PROGRAMS = $(AERPROGRAM) $(AEPROGRAM) $(PROGRAM) - -TESTS = $(AERTEST) $(filter-out %/phist, $(AETEST)) $(TEST) +TESTS = $(filter-out %/phist, $(AETEST)) $(TEST) # TODO: Need a way to take mpiexec tests out of this runTESTS = $(patsubst %,run%,$(TESTS)) @@ -463,15 +410,6 @@ $(AETEST): %: %.o $(LIBRARIES) -L$(builddir)/lib -ltriton \ $(call MODLDFLAGS_AE,$*) $(call MODLIBS_AE,$*) $(LIBS) -$(AERTEST): %: %.o %_module.o $(LIBRARIES) - $(R) " AERTEST" - $(Q) " LD $@" - $(E)$(call MCC_AER,$*) \ - -o $@ $(filter %.o, $^) $(call MODLINKWITH_AER,$*) $(LDFLAGS) \ - -L$(builddir)/lib -ltriton \ - $(call MODLDFLAGS_AER,$*) $(call MODLIBS_AER,$*) \ - $(LIBS) - $(AEPROGRAM): %: %.o $(LIBRARIES) $(R) " AEPROGRAM" $(Q) " LD $@" @@ -481,14 +419,6 @@ $(AEPROGRAM): %: %.o $(LIBRARIES) $(call MODLDFLAGS_AE,$*) $(call MODLIBS_AE,$*) \ $(LIBS) -$(AERPROGRAM): %: %.o $(LIBRARIES) - $(R) " AERPROGRAM" - $(Q) " LD $@" - $(E)$(call MCC_AER,$*) \ - -o $@ $< $(LDFLAGS) \ - -L$(builddir)/lib -ltriton \ - $(call MODLDFLAGS_AER,$*) $(call MODLIBS_AER,$*) $(LIBS) - $(ASGLIBOBJS) $(LIBOBJS) $(TESTOBJS): %.o: %.c $(R) " C OBJECT" $(Q) " CC $@" @@ -506,10 +436,10 @@ $(LIBDEPS) $(TESTDEPS): %.d: %.c $(srcdir)/maint/depend.sh $(call dirname,$*) \ $(LIBCFLAGS) $(CFLAGS) $(call MODCFLAGS_C,$*) $< > $@ -lib/libtriton.a: $(AERLIBOBJS) $(AELIBOBJS) $(LIBOBJS) +lib/libtriton.a: $(AELIBOBJS) $(LIBOBJS) $(Q) " RANLIB $@" $(E)$(INSTALL) -d lib - $(E)ar rcs $@ $(AELIBOBJS) $(LIBOBJS) $(AERLIBOBJS) + $(E)ar rcs $@ $(AELIBOBJS) $(LIBOBJS) lib/libasg.a: $(ASGLIBOBJS) $(Q) " RANLIB $@" @@ -519,19 +449,19 @@ lib/libasg.a: $(ASGLIBOBJS) # shared objects -lib/libtriton.so: $(AERLIBOBJS) $(AELIBOBJS) $(LIBOBJS) +lib/libtriton.so: $(AELIBOBJS) $(LIBOBJS) $(Q) " LDSO $@" $(E)$(INSTALL) -d lib - $(E)$(LDSHARED) -Wl,-soname,libtriton.so -o $@ $(AERLIBOBJS) $(AELIBOBJS) $(LIBOBJS) + $(E)$(LDSHARED) -Wl,-soname,libtriton.so -o $@ $(AELIBOBJS) $(LIBOBJS) -lib/libtriton.dylib: $(AERLIBOBJS) $(AELIBOBJS) $(LIBOBJS) +lib/libtriton.dylib: $(AELIBOBJS) $(LIBOBJS) $(Q) " DYLIB $@" $(E)$(INSTALL) -d lib - $(E)$(TCC) -o lib/libtriton.dylib -dynamiclib -current_version 0.0.1 -install_name lib/libtriton.0.0.1.dylib \ - $(AERLIBOBJS) $(AELIBOBJS) $(LIBOBJS) $(DEPLIBS) + $(E)$(CC) -o lib/libtriton.dylib -dynamiclib -current_version 0.0.1 -install_name lib/libtriton.0.0.1.dylib \ + $(AELIBOBJS) $(LIBOBJS) $(DEPLIBS) # AE rule compiles *.ae files to object files -$(AEOBJS): %.o: $(HSBIN) %.ae +$(AEOBJS): %.o: %.ae $(R) " AEOBJS" $(Q) " AECC $*.ae" $(E) $(AECC) -o $(builddir)/$*.o $(srcdir)/$*.ae \ @@ -550,31 +480,10 @@ $(AEDEPS): %.ae.d: %.ae $(DEPEND_AE_SH) $(LIBCFLAGS) $(CFLAGS) $(call MODCFLAGS_C,$*) \ $< > $@ -DEPEND_AER_SH = $(srcdir)/maint/depend-aer.sh -$(AERDEPS): %.aer.d: %.aer $(DEPEND_AER_SH) - $(R) " AERDEPS" - $(Q) " DEP $@" - $(E)CC=$(call MCC_AER,$*) AESOP=$(AESOP) \ - EXTRAS="$(filter-out $*.o,$(call MODLINKWITH_AER,$*))" \ - $(DEPEND_AER_SH) $(*D) $(srcdir) $(MODNAME_CORE) \ - $(LIBCFLAGS) $(CFLAGS) $(call MODCFLAGS_AER,$*) $< > $@ - -# rule to build remote modules -%_module.o %_module.h %_stubs.hae %.o: %.aer $(HSBIN) - $(R) " REMOTE MODULE" - $(Q) " AERCC $*" - $(E)$(builddir)/maint/aercc -m $(*F) \ - -d $(call RMOD_REGDIR,$*) \ - $< \ - -c $(call MCC_AER,$*) $(VA) $(DA) $(AEPRETTY) \ - -- $(CFLAGS) -I $(AESOP)/include $(LIBCFLAGS) $(CWARNS) \ - $(call MODCFLAGS_AER,$*) - - # Generate header files (%.h) from AE header files # NOTE: this is used by resource .c files that need to include the .h # version of the .hae file that defines the resource interface -$(AESOP_TRHDR): %.h: %.hae $(HSBIN) +$(AESOP_TRHDR): %.h: %.hae $(R) " AESOP HEADER" $(Q) " AECC $*.hae" $(E)$(AECC) -o $(builddir)/$*.h $(srcdir)/$*.hae \ @@ -604,15 +513,6 @@ DEP_PREREQS += $(shell find $(srcdir)/src -name "*.hae") DEP_PREREQS += $(shell find $(srcdir)/src -name "*.h") %.d: $(DEP_PREREQS) -# Compile parsers from haskell source using GHC - -$(HSBIN): %: $(srcdir)/%.lhs $(HSSRC) - $(Q) " GHC $@" - $(E)$(GHC) -O2 -odir $(call canonname,$<) \ - -hidir $(call canonname,$<) \ - -o $@ \ - -main-is $(call modname,$<) \ - -i$(call dir,$<) --make -XFlexibleInstances $< >/dev/null # User targets @@ -676,23 +576,17 @@ INTERM_AESRC=$(patsubst %.o,%.ae.i.s,$(AEOBJS)) INTERM_AEOTHERSRC=$(patsubst %.o,%.ae.i.s,$(AEOBJS)) INTERM_AEINC=$(patsubst %.o,%.ae.i,$(AEOBJS)) INTERM_HAEINC=$(patsubst %.h,%.hae.i,$(AESOP_TRHDR)) -INTERM_REMOTE_MODS=$(foreach mod,$(REMOTE_MODULES),$(mod)_stubs.hae.i,$(mod)_module.ae,$(mod)_module.ae.i,$(mod)_module.ae.i.s) -INTERM_REMOTE_AESRC=$(patsubst %.aer,%.ae.i.s,$(AEROBJS)) -INTERM_REMOTE_AEINC=$(patsubst %.aer,%.ae.i,$(AEROBJS)) INTERM=$(INTERM_AESRC) \ $(INTERM_AEINC) \ $(INTERM_HAEINC) \ - $(INTERM_REMOTE_MODS) \ - $(INTERM_REMOTE_AESRC) \ - $(INTERM_REMOTE_AEINC) \ $(INTERM_AEOTHERSRC) clean:: $(Q) " CLEAN" - $(E)rm -f $(AEROBJS) $(AEOBJS) $(LIBOBJS) \ - $(AERTEST) $(AETEST) $(TEST) \ - $(AERPROGRAM) $(AEPRROGRAM) $(PROGRAM) \ + $(E)rm -f $(AEOBJS) $(LIBOBJS) \ + $(AETEST) $(TEST) \ + $(AEPROGRAM) $(PROGRAM) \ $(AESOP_TRHDR) \ $(LIBRARIES) $(DEPENDS) $(TEST_RESULTS) $(TEST_OUT) \ $(INTERM) \ @@ -702,14 +596,10 @@ clean_test_progs:: $(Q) " CLEAN TEST PROGRAMS" $(E) rm -fv $(TEST_PROGS) -parserclean:: - $(Q) " PARSER CLEAN" - $(E)rm -f $(HSBIN) $(HSOBJ) $(HSINT) - # this is where we include all of our automatic dependencies. # NOTE: we wrap this in ifneq's in order to prevent the dependencies # from being generated for special targets that don't require them -ifeq (,$(filter clean parserclean nodep,$(MAKECMDGOALS))) +ifeq (,$(filter clean nodep,$(MAKECMDGOALS))) -include $(DEPENDS) endif # add this as a make goal to disable rebuilding deps diff --git a/code/configure.ac b/code/configure.ac index a798411..5f284df 100644 --- a/code/configure.ac +++ b/code/configure.ac @@ -41,13 +41,12 @@ AC_MSG_RESULT(yes) AC_SYS_LARGEFILE -AX_GHC("") - USE_AESOP=UNKNOWN AC_ARG_WITH(aesop, [ --with-aesop=<dir> Location of external Aesop package], [ - AC_CHECK_FILE(${withval}/aesop.h) + AC_CHECK_FILE(${withval}/include/aesop/aesop.h,[], + [AC_MSG_ERROR([Could not find aesop])]) USE_AESOP=${withval} ], [ @@ -61,7 +60,8 @@ USE_MERCURY= AC_ARG_WITH(mercury, [ --with-mercury=<dir> Location of external mercury package], [ - AC_CHECK_FILE(${withval}/mercury.h) + AC_CHECK_FILE(${withval}/include/mercury.h,[], + [AC_MSG_ERROR([Could not find mercury])]) USE_MERCURY=${withval} ], [ @@ -156,108 +156,6 @@ fi dnl todo: add check making sure that if the user manually enabled perftools dnl and we cannot find them we complain - - -dnl ========================================================================= -dnl == SSM =================================================== -dnl ========================================================================= - -AC_ARG_WITH(ssm, - AS_HELP_STRING([--with-ssm=dir], - [Location of ssm installation]), - SSM_DIR="$withval",SSM_DIR="") - -echo "SSM_DIR=${SSM_DIR}" - -#if test -n "$SSM_DIR" ; -#then - # echo "Setting flags for ssm" - CPPFLAGS="$CPPFLAGS -I${SSM_DIR}/include" - LDFLAGS="$LDFLAGS -L${SSM_DIR}/lib" -#fi - -BUILD_SSM=1 - -AC_CHECK_HEADERS([ssm.h],[], - [BUILD_SSM=]) - -AC_CHECK_LIB([ssm],[ssm_stop],[],[BUILD_SSM=],[-lpthread -lssmptcp]) -AC_CHECK_LIB([ssmptcp],[ssmptcp_new_tp],[],[BUILD_SSM=],[-lssm -lpthread]) - -AC_MSG_CHECKING([if SSM support will be enabled]) -if test "x${BUILD_SSM}" == "x1"; then -AC_MSG_RESULT([yes]) -else -AC_MSG_RESULT([no]) -AC_MSG_WARN([Please specify a valid --with-ssm to enable SSM]) -fi -AC_SUBST(BUILD_SSM) - -dnl ====================================================================== -dnl Look for MPI; Auto-enable if possible, unless disabled by the user -dnl ====================================================================== -AC_ARG_WITH([mpi],[AS_HELP_STRING([--with-mpi@<:@=DIR@:>@], - [location of MPI library])], - [], - [with_mpi=check]) - -BUILD_MPI= -AS_IF([test "x$with_mpi" != "xno"], -[ - if test "x$with_mpi" != "xcheck" -a "x$with_mpi" != "xyes"; then - checkpath="${with_mpi}/bin" - else - checkpath="$PATH" - fi - AC_CHECK_PROGS(MPICC, [mpicc], [], [$checkpath]) - # - # with mpich2-1.3, mpiCC invokes gcc... not g++ - # - AC_CHECK_PROGS(MPICXX, [mpic++ mpicxx mpiCC], [], [$checkpath]) - if test "x$with_mpi" != "xcheck"; then - if test -z "${MPICXX}" ; then - AC_MSG_ERROR("--with-mpi given but could not find mpic++!") - fi - if test -z "${MPICC}" ; then - AC_MSG_ERROR("--with-mpi given but could not find mpicc!") - fi - fi - ]) - -if test "${MPICXX}x" != "x" ; then - if test "${MPICC}x" == "x"; then - AC_MSG_ERROR("Found MPI C++ compiler but not MPI C compiler??") - fi - dnl - dnl make sure mpi compiler path is a full path - dnl - if test "x$with_mpi" != "xcheck" -a "x$with_mpi" != "xyes"; then - MPICXX="${checkpath}/${MPICXX}" - fi - - if test "x$with_mpi" != "xcheck" -a "x$with_mpi" != "xyes"; then - MPICC="${checkpath}/${MPICC}" - fi - BUILD_MPI=1 -fi - -AC_MSG_CHECKING([if MPI support will be enabled]) -if test "x${BUILD_MPI}" == "x1"; then -AC_MSG_RESULT([yes]) -else -AC_MSG_RESULT([no]) -fi - -MPICFLAGS="" -MPILDFLAGS="" -MPILIBS="" - -AC_SUBST(MPICC) -AC_SUBST(MPICFLAGS) -AC_SUBST(MPILDFLAGS) -AC_SUBST(MPILIBS) -AC_SUBST(BUILD_MPI) - dnl check for unix variants AX_UNIX("") @@ -383,13 +281,7 @@ AC_SUBST(MERCURY_CFLAGS) AC_CHECK_LIB([crypto], [SHA1], [], [AC_MSG_ERROR([-lcrypto required. Install OpenSSL.])]) -# Use aecc and aecc-utils in Aesop distribution -# AC_CONFIG_FILES([maint/aecc-utils], [chmod +x maint/aecc-utils]) -# AC_CONFIG_FILES([maint/aecc], [chmod +x maint/aecc]) -AC_CONFIG_FILES([maint/aercc], [chmod +x maint/aercc]) - AC_OUTPUT([Makefile -src/aesop/parser/module.mk src/common/module.mk src/common/tests/module.mk src/common/resources/scheduling/module.mk diff --git a/code/maint/aercc.in b/code/maint/aercc.in deleted file mode 100644 index cd31157..0000000 --- a/code/maint/aercc.in +++ /dev/null @@ -1,211 +0,0 @@ -#!/bin/bash - -SRCDIR=@SRC_RELATIVE_TOP@ -BUILDDIR=@BUILD_ABSOLUTE_TOP@ -AESOP=@USE_AESOP@ - -AERPARSER=${BUILDDIR}/src/aesop/parser/ae-remote-parser - -source ${AESOP}/bin/aecc-utils - -# SRCDIR is used in sed patterns: escape special characters -if test "${SRCDIR}" = ".."; then - SRCDIR="\\.\\." -elif test "${SRCDIR}" = "."; then - SRCDIR="\\." -fi - -setUsageMessage "Usage: aercc <options> <file> -- <compiler options>. Specify --help for a list of options." - -setHelpStart "Usage: aercc <options> <file> -- <compiler options>.\n" -setHelpEnd "\tAll other options (after --) are passed on to the compiler.\n" - -addOption h help no none no empty printHelpAndExit "This message." -addOption c compiler yes compiler no "<compiler path>" none "Path to the compiler to use. Defaults to gcc." -addOption p pretty no none no empty none "Make the parser output pretty code (no source lines)." -addOption s "just-source" no none no empty none "Just generate C source and header files." -addOption v "show-commands" no none no empty none "Show commands run." -addOption S "from-interm" no none no empty none "Start from (existing) intermediate files." -addOption M "generate-deps" no none no empty none "Generate dependency file during preprocessing step." -addOption l "pre-compile" no none no empty none "Pre-compile aesop for syntax checking." -addOption m "module-name" yes servicename no "<module name>" none "Specifies the service name to use for the RPC module." -addOption d "registry-dir" yes registrydir no "<registry dir>" none "Specifies the registry directory to place generated module files." - -compiler=gcc -parseropts="" -outputfile=a.out - -parseOpts $@ - -clean= -addToClean() -{ - if test "${hasOpt_pretty}" != "yes"; then - clean="${clean} $1" - fi -} - -exitIfFailed() -{ - res=$1 - if test $res -ne 0; then - doClean ${clean} - exit $1 - fi -} - -f=${registrydir}/${servicename} -fullservicename=$(echo ${f} | sed -e "s|//|/|g") - -if test "${hasOpt_pretty}" = "yes"; then - parseropts="${parseropts} --pretty" -fi - -verbose=no -if test "${hasOpt_show_commands}" = "yes"; then - verbose=yes -fi - -bparserout= -modcout= -if test ${hasOpt_from_interm} != "yes"; then - - - if test -z "${INPUTFILES}"; then - echo "No input file specified!\n" - showUsage 1 - fi - - poutfiles= - cpp_inputs= - for ifile in ${INPUTFILES}; do - - # take .o and replace src directory with nothing (we should be in the build dir) - buildifile=$(echo "${ifile}" | sed -e "s|^ *${SRCDIR}||" | sed -e "s|^/||") - - cppout=${buildifile}.i - addToClean ${cppout} - - printVerbose "COMPILEROPTS: ${COMPILEROPTS}" - - # Step 1: first generate preprocessed file and dep file - doCPP ${verbose} ${compiler} ${ifile} ${cppout} ${outputfile} "${hasOpt_pre_compile}" ${COMPILEROPTS} - - exitIfFailed $? - - cpp_inputs="${cpp_inputs} ${cppout}" - - # Step 2: next run parser on preprocessed file - poutfile=${buildifile}.r - addToClean ${poutfile} - - printVerbose "parseRemoteSource..." - parseRemoteSource ${servicename} ${registrydir} ${verbose} ${compiler} ${cppout} ${poutfile} "${ifile}" "${parseropts}" ${COMPILEROPTS} - RC=$? - printVerbose "parseRemoteSource done. RC=$RC" - exitIfFailed $RC - -# remote_cppout=${buildifile}.r.i -# addToClean ${remote_cppout} - - # parse generated .r - bparserinput=${poutfile} - bparserout=${poutfile}.s - if test "${hasOpt_just_source}" != "yes"; then addToClean ${bparserout}; fi - - parseAesopSource ${verbose} ${compiler} ${bparserinput} ${bparserout} "${ifile}" "${parseropts}" ${COMPILEROPTS} - exitIfFailed $? - - compilerinputs="${compilerinputs} ${bparserout}" - done - - # generates: <servicename>_module.ae (aesop source file) - # <servicename>_module.h (C header) - # <servicename>_stubs.hae (aesop header) - - if test -z "${servicename}"; then - echo "No service name specified!" - showUsage 1 - fi - if test -z "${registrydir}"; then - echo "No registry directory specified!" - showUsage 1 - fi - - parseRemoteSource ${servicename} ${registrydir} ${verbose} ${compiler} "${cpp_inputs}" none none "${parseropts}" ${COMPILEROPTS} - - addToClean ${fullservicename}_module.ae - - # parse generated _module.ae - modfile=${fullservicename}_module.ae - modcppout=${modfile}.i - addToClean ${modcppout} - doCPP ${verbose} ${compiler} ${modfile} ${modcppout} ${modfile} no ${COMPILEROPTS} - exitIfFailed $? - - modcout=${modfile}.s - if test "${hasOpt_just_source}" != "yes"; then addToClean ${modcout}; fi - parseAesopSource ${verbose} ${compiler} ${modcppout} ${modcout} ${modfile} none ${COMPILEROPTS} - exitIfFailed $? - - # parse generated _stubs.hae - - stubfile=${fullservicename}_stubs.hae - stubcppout=${stubfile}.i - addToClean ${stubcppout} - - doCPP ${verbose} ${compiler} ${stubfile} ${stubcppout} ${stubfile} no ${COMPILEROPTS} - exitIfFailed $? - - stubhout=${fullservicename}_stubs.h - - parseAesopHeader ${verbose} ${compiler} ${stubcppout} ${stubhout} ${stubfile} none ${COMPILEROPTS} - exitIfFailed $? - - # if only sources were supposed to be generated, stop here - if test "${hasOpt_just_source}" = "yes" ; then - rm -f ${clean} - exit 0 - fi -else - modcout=${fullservicename}_module.s - compilerinputs="${INPUTFILES}" -fi - -# compile -objfiles= -for cf in ${compilerinputs}; do - if test "${hasOpt_from_interm}" != "yes"; then - # we're not building from intermediate, so we need to take from the input files - # take the first input file (peak the head) - if=$(echo ${INPUTFILES} | sed -e "s|^$[^ ]*$ .*|\1|") - # this removes the first file from the list (pop the head) - INPUTFILES=$(echo ${INPUTFILES} | sed -e "s|^[^ ]*\ $.*$$|\1|") - # convert input file to .o - outsrc=$(echo "${if}" | sed -e "s|\.[^.]*$|.o|") - else - # convert input file to .o - outsrc=$(echo "${cf}" | sed -e "s|\.s$|.o|") - fi - # take .o and replace src directory with nothing (we should be in the build dir) - outbuild=$(echo "${outsrc}" | sed -e "s|^${SRCDIR}||" | sed -e "s|^/||") - - doCompile ${verbose} ${compiler} ${cf} ${outbuild} ${COMPILEROPTS} - exitIfFailed $? - objfiles="${objfiles} ${outbuild}" -done - -modobj=${fullservicename}_module.o - -# make dependency file -#echo "${modobj} ${objfiles}: ${INPUTFILES}" > ${fullservicename}.d - -# compile generated _module.s -doCompile ${verbose} ${compiler} ${modcout} ${modobj} ${COMPILEROPTS} -exitIfFailed $? - -rm -f ${clean} - -exit 0 - -# Done! diff --git a/code/maint/config/ghc.m4 b/code/maint/config/ghc.m4 deleted file mode 100644 index c3933bb..0000000 --- a/code/maint/config/ghc.m4 +++ /dev/null @@ -1,26 +0,0 @@ -AC_DEFUN([AX_CHECK_GHCPKG], -[ - GHCPKG=$1 - pkgname=$2 - AC_MSG_CHECKING([for valid ghc $pkgname module..]) - res=`$GHCPKG find-module --simple-output $pkgname` - if test -z "$res"; then - AC_MSG_ERROR(no. Please run the scripts setup-hs-local and setup-langc in maint/hs/.) - fi - AC_MSG_RESULT(yes) -]) - -AC_DEFUN([AX_GHC], -[ - AC_ARG_WITH(ghc, - [ --with-ghc=<dir> Location of installed ghc], - [AC_PATH_PROG(GHC, ${withval}/bin/ghc) - AC_PATH_PROG(GHCPKG, ghc-pkg, ${withval}/bin/ghc-pkg)], - [AC_PATH_PROG(GHC, ghc) - AC_PATH_PROG(GHCPKG, ghc-pkg)]) - AC_SUBST(GHC) - AC_SUBST(GHCPKG) - - AX_CHECK_GHCPKG($GHCPKG, Language.C) - AX_CHECK_GHCPKG($GHCPKG, Text.Regex.PCRE) -]) diff --git a/code/maint/depend-aer.sh b/code/maint/depend-aer.sh deleted file mode 100755 index 227aa5e..0000000 --- a/code/maint/depend-aer.sh +++ /dev/null @@ -1,23 +0,0 @@ -#!/bin/sh -DIR="$1" -shift -SRCDIR="$1" -shift -if [ -n "$DIR" ] ; then - DIR="$DIR"/ -fi -RMOD="$1" -shift - -# Explanation of some arguments: -# -MG tells gcc to ignore errors from missing header files, which is -# important in Triton because some of them are autogenerated and won't exist -# yet when we run this tool. -# -MM tells gcc not to bother including system header files. Our -# build is slow enough already so it isn't worth it. -# -x c tells gcc to pretend like the input file is a c file regardless of -# filename extension. This is useful for cases where we want gcc to -# generate dependencies on raw .ae files. -# there is one manual addition to the dependencies (ae-remote-parser.h) -# which is implicitly added by the aesop remote parser -exec $CC -x c -M -MM -MG "$@" | sed -e "s@^$.*$\.o:@$DIR\1.o $DIR\1_module.o $DIR\1_stubs.hae $DIR\1_module.h: src/aesop/ae-remote-parser.h $AESOP/include/aesop/ae-blocking-parser.h \\\\= $EXTRAS \\\\=@" | tr "=" "\n" diff --git a/code/maint/hs/.gitignore b/code/maint/hs/.gitignore deleted file mode 100644 index 48355d8..0000000 --- a/code/maint/hs/.gitignore +++ /dev/null @@ -1,9 +0,0 @@ -GHC -alex-2.3.1 -build -happy-1.18.4 -mtl-1.1.0.2 -regex-base-0.93.1 -regex-pcre-builtin-0.94.2.1.7.7 -regex-posix-0.94.1 -syb-0.1.0.1 diff --git a/code/maint/hs/README.txt b/code/maint/hs/README.txt deleted file mode 100644 index 53b70c0..0000000 --- a/code/maint/hs/README.txt +++ /dev/null @@ -1,2 +0,0 @@ - -See src/aesop/parser/README.txt diff --git a/code/maint/hs/language-c-0.4.2.patch b/code/maint/hs/language-c-0.4.2.patch deleted file mode 100644 index 77dfc24..0000000 --- a/code/maint/hs/language-c-0.4.2.patch +++ /dev/null @@ -1,309 +0,0 @@ -diff -Naur language-c-0.4.2/language-c.cabal language-c-0.4.2-new/language-c.cabal ---- language-c-0.4.2/language-c.cabal 2011-08-24 09:51:22.000000000 -0500 -+++ language-c-0.4.2-new/language-c.cabal 2012-05-26 11:43:57.893087938 -0500 -@@ -1,4 +1,4 @@ --Name: language-c -+Name: language-aesop - Version: 0.4.2 - Cabal-Version: >= 1.6 - Build-Type: Simple -@@ -6,7 +6,7 @@ - License-File: LICENSE - Copyright: LICENSE - Author: AUTHORS --Maintainer: benedikt.huber(a)gmail.com -+Maintainer: benedikt.huber(a)gmail.com [Modified by Sam Lang (slang(a)mcs.anl.gov) for aesop] - Stability: experimental - Homepage: http://www.sivity.net/projects/language.c/ - Bug-reports: http://www.sivity.net/projects/language.c/ -@@ -15,6 +15,7 @@ - Description: Language C is a haskell library for the analysis and generation of C code. - It features a complete, well tested parser and pretty printer for all of C99 and a large - set of GNU extensions. -+ This modified version includes extensions for AESOP. - Category: Language - - Extra-Source-Files: AUTHORS AUTHORS.c2hs ChangeLog README -diff -Naur language-c-0.4.2/src/Language/C/Data/Ident.hs language-c-0.4.2-new/src/Language/C/Data/Ident.hs ---- language-c-0.4.2/src/Language/C/Data/Ident.hs 2011-08-24 09:51:19.000000000 -0500 -+++ language-c-0.4.2-new/src/Language/C/Data/Ident.hs 2012-05-26 11:45:06.753087732 -0500 -@@ -17,7 +17,7 @@ - module Language.C.Data.Ident ( - Ident(..), - SUERef(..), isAnonymousRef, -- mkIdent, builtinIdent, internalIdent, internalIdentAt, isInternalIdent, identToString, dumpIdent) -+ mkIdent, builtinIdent, internalIdent, internalIdentAt, isInternalIdent, identToString, dumpIdent, hashIdent) - where - - -- TODO (comment from manuel): -@@ -133,3 +133,6 @@ - -- | dump the identifier string and its positions for debugging purposes - dumpIdent :: Ident -> String - dumpIdent ide = identToString ide ++ " at " ++ show (nodeInfo ide) -+ -+hashIdent :: Ident -> Int -+hashIdent (Ident _ h _) = h -diff -Naur language-c-0.4.2/src/Language/C/Parser/Lexer.x language-c-0.4.2-new/src/Language/C/Parser/Lexer.x ---- language-c-0.4.2/src/Language/C/Parser/Lexer.x 2011-08-24 09:51:19.000000000 -0500 -+++ language-c-0.4.2-new/src/Language/C/Parser/Lexer.x 2012-05-26 11:46:33.213087517 -0500 -@@ -358,7 +358,14 @@ - idkwtok ('v' : 'o' : 'l' : 'a' : 't' : 'i' : 'l' : 'e' : []) = tok 8 CTokVolatile - idkwtok ('_' : '_' : 'v' : 'o' : 'l' : 'a' : 't' : 'i' : 'l' : 'e' : '_' : '_' : []) = tok 12 CTokVolatile - idkwtok ('w' : 'h' : 'i' : 'l' : 'e' : []) = tok 5 CTokWhile -- -+idkwtok ('_':'_':'b':'l':'o':'c':'k':'i':'n':'g':[]) = tok 10 CTokBlocking -+idkwtok ('_':'_':'r':'e':'m':'o':'t':'e':[]) = tok 10 CTokRemote -+idkwtok ('p':'w':'a':'i':'t':[]) = tok 5 CTokPWait -+idkwtok ('p':'b':'r':'a':'n':'c':'h':[]) = tok 7 CTokPBranch -+idkwtok ('p':'b':'r':'e':'a':'k':[]) = tok 6 CTokPBreak -+idkwtok ('p':'p':'r':'i':'v':'a':'t':'e':[]) = tok 6 CTokPPrivate -+idkwtok ('p':'s':'h':'a':'r':'e':'d':[]) = tok 6 CTokPShared -+ - idkwtok cs = \pos -> do - name <- getNewName - let len = case length cs of l -> l -diff -Naur language-c-0.4.2/src/Language/C/Parser/Parser.y language-c-0.4.2-new/src/Language/C/Parser/Parser.y ---- language-c-0.4.2/src/Language/C/Parser/Parser.y 2011-08-24 09:51:19.000000000 -0500 -+++ language-c-0.4.2-new/src/Language/C/Parser/Parser.y 2012-05-26 11:51:02.101086845 -0500 -@@ -217,6 +217,8 @@ - typedef { CTokTypedef _ } - typeof { CTokTypeof _ } - "__thread" { CTokThread _ } -+"__blocking" { CTokBlocking _ } -+"__remote" { CTokRemote _ } - union { CTokUnion _ } - unsigned { CTokUnsigned _ } - void { CTokVoid _ } -@@ -236,7 +238,12 @@ - "__builtin_va_arg" { CTokGnuC GnuCVaArg _ } - "__builtin_offsetof" { CTokGnuC GnuCOffsetof _ } - "__builtin_types_compatible_p" { CTokGnuC GnuCTyCompat _ } -- -+pwait { CTokPWait _ } -+pbranch { CTokPBranch _ } -+pbreak { CTokPBreak _ } -+pshared { CTokPShared _ } -+pprivate { CTokPPrivate _ } -+ - %% - - -@@ -359,6 +366,7 @@ - | iteration_statement { $1 } - | jump_statement { $1 } - | asm_statement {% withNodeInfo $1 (CAsm $1) } -+ | parallel_statement { $1 } - - - -- parse C labeled statement (C99 6.8.1) -@@ -374,7 +382,19 @@ - {% withNodeInfo $1 $ CCases $2 $4 $6 } - - ---- parse C compound statement (C99 6.8.2) -+-- parse Parallel C statement -+-- -+parallel_statement :: { CStat } -+parallel_statement -+ : pwait statement -+ {% withNodeInfo $1 $ CPWait $2 } -+ | pbranch statement -+ {% withNodeInfo $1 $ CPBranch $2 } -+ | pbreak -+ {% withNodeInfo $1 $ CPBreak } -+ -+ -+ -- parse C compound statement (C99 6.8.2) - -- - -- * GNU extension: '__label__ ident;' declarations - -- -@@ -550,7 +570,7 @@ - -- storage class and type qualifier - --------------------------------------------------------------------------------------------------------------- - attr :- __attribute__((..)) --storage_class :- typedef | extern | static | auto | register | __thread -+storage_class :- typedef | extern | static | auto | register | __thread | __blocking | pprivate | ppshared - type_qualifier :- const | volatile | restrict | inline - type_qualifier_list :- type_qualifier+ - -@@ -755,7 +775,7 @@ - - - -- A mixture of type qualifiers (const, volatile, restrict, inline) and storage class specifiers ---- (extern, static, auto, register, __thread), in any order, but containing at least one storage class specifier. -+-- (extern, static, auto, register, __thread, __blocking, pshared, pprivate), in any order, but containing at least one storage class specifier. - -- - -- declaration_qualifier_list :- <permute> type_qualifier* storage_class+ - -- -@@ -1188,6 +1208,10 @@ - | volatile {% withNodeInfo $1 $ CVolatQual } - | restrict {% withNodeInfo $1 $ CRestrQual } - | inline {% withNodeInfo $1 $ CInlineQual } -+ | "__blocking" {% withNodeInfo $1 $ CBlocking } -+ | "__remote" {% withNodeInfo $1 $ CRemote } -+ | pshared {% withNodeInfo $1 $ CPShared } -+ | pprivate {% withNodeInfo $1 $ CPPrivate } - - -- a list containing at least one type_qualifier (const, volatile, restrict, inline) - -- and additionally CAttrs -@@ -2161,13 +2185,17 @@ - withAsmNameAttrs (mAsmName, newAttrs) declr = setAsmName mAsmName (appendObjAttrsR newAttrs declr) - - appendDeclrAttrs :: [CAttr] -> CDeclrR -> CDeclrR --appendDeclrAttrs newAttrs (CDeclrR ident (Reversed []) asmname cattrs at) -- = CDeclrR ident empty asmname (cattrs ++ newAttrs) at - appendDeclrAttrs newAttrs (CDeclrR ident (Reversed (x:xs)) asmname cattrs at) -- = CDeclrR ident (Reversed (appendAttrs x : xs)) asmname cattrs at where -- appendAttrs (CPtrDeclr typeQuals at) = CPtrDeclr (typeQuals ++ map CAttrQual newAttrs) at -- appendAttrs (CArrDeclr typeQuals arraySize at) = CArrDeclr (typeQuals ++ map CAttrQual newAttrs) arraySize at -- appendAttrs (CFunDeclr parameters cattrs at) = CFunDeclr parameters (cattrs ++ newAttrs) at -+ | hasFunDeclr (x:xs) = CDeclrR ident (Reversed (appendAttrs x : xs)) asmname cattrs at -+ | otherwise = CDeclrR ident (Reversed (x:xs)) asmname (cattrs ++ newAttrs) at -+ where appendAttrs (CPtrDeclr typeQuals at) = CPtrDeclr (typeQuals ++ map CAttrQual newAttrs) at -+ appendAttrs (CArrDeclr typeQuals arraySize at) = CArrDeclr (typeQuals ++ map CAttrQual newAttrs) arraySize at -+ appendAttrs (CFunDeclr parameters cattrs at) = CFunDeclr parameters (cattrs ++ newAttrs) at -+ isFunDeclr (CFunDeclr _ _ _) = True -+ isFunDeclr _ = False -+ hasFunDeclr ds = any isFunDeclr ds -+appendDeclrAttrs newAttrs (CDeclrR ident derived asmname cattrs at) -+ = CDeclrR ident derived asmname (cattrs ++ newAttrs) at - - ptrDeclr :: CDeclrR -> [CTypeQual] -> NodeInfo -> CDeclrR - ptrDeclr (CDeclrR ident derivedDeclrs asmname cattrs dat) tyquals at -diff -Naur language-c-0.4.2/src/Language/C/Parser/Tokens.hs language-c-0.4.2-new/src/Language/C/Parser/Tokens.hs ---- language-c-0.4.2/src/Language/C/Parser/Tokens.hs 2011-08-24 09:51:19.000000000 -0500 -+++ language-c-0.4.2-new/src/Language/C/Parser/Tokens.hs 2012-05-26 11:52:19.589086751 -0500 -@@ -132,7 +132,14 @@ - | CTokTyIdent !PosLength !Ident -- `typedef-name' identifier - | CTokGnuC !GnuCTok !PosLength -- special GNU C tokens - | CTokEof -- end of file -- -+ | CTokBlocking !PosLength -- '__blocking' -+ | CTokRemote !PosLength -- '__remote' -+ | CTokPWait !PosLength -- `pwait' -+ | CTokPBranch !PosLength -- `pbranch' -+ | CTokPBreak !PosLength -- `pbreak' -+ | CTokPShared !PosLength -- `pshared' -+ | CTokPPrivate !PosLength -- `pprivate' -+ - -- special tokens used in GNU C extensions to ANSI C - -- - data GnuCTok = GnuCAttrTok -- `__attribute__' -@@ -242,6 +249,13 @@ - posLenOfTok (CTokIdent pos _) = pos - posLenOfTok (CTokTyIdent pos _) = pos - posLenOfTok (CTokGnuC _ pos ) = pos -+posLenOfTok (CTokBlocking pos ) = pos -+posLenOfTok (CTokRemote pos ) = pos -+posLenOfTok (CTokPWait pos ) = pos -+posLenOfTok (CTokPBranch pos ) = pos -+posLenOfTok (CTokPBreak pos ) = pos -+posLenOfTok (CTokPPrivate pos ) = pos -+posLenOfTok (CTokPShared pos ) = pos - posLenOfTok CTokEof = error "tokenPos: Eof" - - instance Show CToken where -@@ -327,11 +341,18 @@ - showsPrec _ (CTokTypedef _ ) = showString "typedef" - showsPrec _ (CTokTypeof _ ) = showString "typeof" - showsPrec _ (CTokThread _ ) = showString "__thread" -+ showsPrec _ (CTokBlocking _ ) = showString "__blocking" -+ showsPrec _ (CTokRemote _ ) = showString "__remote" - showsPrec _ (CTokUnion _ ) = showString "union" - showsPrec _ (CTokUnsigned _ ) = showString "unsigned" - showsPrec _ (CTokVoid _ ) = showString "void" - showsPrec _ (CTokVolatile _ ) = showString "volatile" - showsPrec _ (CTokWhile _ ) = showString "while" -+ showsPrec _ (CTokPWait _ ) = showString "pwait" -+ showsPrec _ (CTokPBranch _ ) = showString "pbranch" -+ showsPrec _ (CTokPBreak _ ) = showString "pbreak" -+ showsPrec _ (CTokPPrivate _ ) = showString "pprivate" -+ showsPrec _ (CTokPShared _ ) = showString "pshared" - showsPrec _ (CTokCLit _ c) = shows c - showsPrec _ (CTokILit _ i) = shows i - showsPrec _ (CTokFLit _ f) = shows f -diff -Naur language-c-0.4.2/src/Language/C/Pretty.hs language-c-0.4.2-new/src/Language/C/Pretty.hs ---- language-c-0.4.2/src/Language/C/Pretty.hs 2011-08-24 09:51:19.000000000 -0500 -+++ language-c-0.4.2-new/src/Language/C/Pretty.hs 2012-05-26 11:53:45.233086438 -0500 -@@ -154,7 +154,11 @@ - pretty (CReturn Nothing _) = ii $ text "return" <> semi - pretty (CReturn (Just e) _) = ii $ text "return" <+> pretty e <> semi - pretty (CAsm asmStmt _) = pretty asmStmt -- prettyPrec p (CCompound localLabels bis _) = -+ pretty (CPWait stmt _) = -+ ii $ text "pwait" $+$ prettyPrec (-1) stmt -+ pretty (CPBranch stmt _) = ii $ text "pbranch" $+$ prettyPrec (-1) stmt -+ pretty (CPBreak _) = ii $ text "pbreak" <> semi -+ prettyPrec p (CCompound localLabels bis ni) = - let inner = text "{" $+$ mlistP ppLblDecls localLabels $+$ vcat (map pretty bis) $$ text "}" - in if p == -1 then inner else ii inner - where ppLblDecls = vcat . map (\l -> text "__label__" <+> identP l <+> semi) -@@ -253,7 +257,11 @@ - pretty (CRestrQual _) = text "__restrict" - pretty (CInlineQual _) = text "inline" - pretty (CAttrQual a) = attrlistP [a] -- -+ pretty (CBlocking _) = text "__blocking" -+ pretty (CRemote _) = text "__remote" -+ pretty (CPShared _) = text "pshared" -+ pretty (CPPrivate _) = text "pprivate" -+ - instance Pretty CStructUnion where - pretty (CStruct tag ident Nothing cattrs _) = pretty tag <+> attrlistP cattrs <+> maybeP identP ident - pretty (CStruct tag ident (Just []) cattrs _) = -@@ -375,8 +383,8 @@ - <+> maybeP pretty expr2 <+> text ":" <+> prettyPrec 4 expr3 - prettyPrec p (CBinary op expr1 expr2 _) = - let prec = binPrec op -- in parenPrec p prec $ prettyPrec prec expr1 -- <+> pretty op <+> prettyPrec (prec + 1) expr2 -+ in parenPrec p prec $ text "(" <+> prettyPrec prec expr1 -+ <+> pretty op <+> prettyPrec (prec + 1) expr2 <+> text ")" - prettyPrec p (CCast decl expr _) = - parenPrec p 25 $ text "(" <> pretty decl <> text ")" - <+> prettyPrec 25 expr -diff -Naur language-c-0.4.2/src/Language/C/Syntax/AST.hs language-c-0.4.2-new/src/Language/C/Syntax/AST.hs ---- language-c-0.4.2/src/Language/C/Syntax/AST.hs 2011-08-24 09:51:19.000000000 -0500 -+++ language-c-0.4.2-new/src/Language/C/Syntax/AST.hs 2012-05-26 11:55:23.797086194 -0500 -@@ -295,6 +295,9 @@ - | CReturn (Maybe (CExpression a)) a - -- | assembly statement - | CAsm (CAssemblyStatement a) a -+ | CPWait (CStatement a) a -+ | CPBranch (CStatement a) a -+ | CPBreak a - deriving (Show, Data,Typeable {-! , CNode , Annotated !-}) - - -- Derived instance relies on fmap2 :( -@@ -453,6 +456,10 @@ - | CRestrQual a - | CInlineQual a - | CAttrQual (CAttribute a) -+ | CBlocking a -- ^ blocking function -+ | CRemote a -- ^ remote function -+ | CPShared a -- ^ shared variable across pbranch blocks -+ | CPPrivate a -- ^ private variable across pbranch blocks - deriving (Show, Data,Typeable {-! ,CNode ,Functor ,Annotated !-}) - - -@@ -857,6 +864,9 @@ - nodeInfo (CBreak d) = nodeInfo d - nodeInfo (CReturn _ n) = nodeInfo n - nodeInfo (CAsm _ n) = nodeInfo n -+ nodeInfo (CPWait _ t) = nodeInfo t -+ nodeInfo (CPBranch _ t) = nodeInfo t -+ nodeInfo (CPBreak t) = nodeInfo t - - instance (CNode t1) => Pos (CStatement t1) where - posOf x = posOf (nodeInfo x) -@@ -1102,6 +1112,10 @@ - nodeInfo (CRestrQual d) = nodeInfo d - nodeInfo (CInlineQual d) = nodeInfo d - nodeInfo (CAttrQual d) = nodeInfo d -+ nodeInfo (CBlocking d) = nodeInfo d -+ nodeInfo (CRemote d) = nodeInfo d -+ nodeInfo (CPPrivate d) = nodeInfo d -+ nodeInfo (CPShared d) = nodeInfo d - - instance (CNode t1) => Pos (CTypeQualifier t1) where - posOf x = posOf (nodeInfo x) diff --git a/code/maint/hs/language-c-0.4.tar.gz b/code/maint/hs/language-c-0.4.tar.gz deleted file mode 100644 index 505521f..0000000 Binary files a/code/maint/hs/language-c-0.4.tar.gz and /dev/null differ diff --git a/code/maint/hs/language-c.patch b/code/maint/hs/language-c.patch deleted file mode 100644 index fffc133..0000000 --- a/code/maint/hs/language-c.patch +++ /dev/null @@ -1,317 +0,0 @@ -diff -Naur language-c-0.4/language-c.cabal language-c-0.4-new/language-c.cabal ---- language-c-0.4/language-c.cabal 2011-01-06 11:46:35.000000000 -0600 -+++ language-c-0.4-new/language-c.cabal 2011-01-06 15:31:57.629238232 -0600 -@@ -1,4 +1,4 @@ --Name: language-c -+Name: language-aesop - Version: 0.4 - Cabal-Version: >= 1.2 - Build-Type: Simple -@@ -6,7 +6,7 @@ - License-File: LICENSE - Copyright: LICENSE - Author: AUTHORS --Maintainer: benedikt.huber(a)gmail.com -+Maintainer: benedikt.huber(a)gmail.com [Modified by Sam Lang (slang(a)mcs.anl.gov) for aesop] - Stability: experimental - Homepage: http://www.sivity.net/projects/language.c/ - Bug-reports: http://www.sivity.net/projects/language.c/ -@@ -14,6 +14,7 @@ - Description: Language C is a haskell library for the analysis and generation of C code. - It features a complete, well tested parser and pretty printer for all of C99 and a large - set of GNU extensions. -+ This modified version includes extensions for AESOP. - Category: Language - - Extra-Source-Files: AUTHORS AUTHORS.c2hs ChangeLog README -@@ -48,7 +49,6 @@ - Build-Depends: bytestring >= 0.9.0 - else - cpp-options: -DNO_BYTESTRING -- Build-Tools: happy, alex - - Hs-Source-Dirs: src - Exposed-Modules: -diff -Naur language-c-0.4/src/Language/C/Data/Ident.hs language-c-0.4-new/src/Language/C/Data/Ident.hs ---- language-c-0.4/src/Language/C/Data/Ident.hs 2011-01-06 11:46:35.000000000 -0600 -+++ language-c-0.4-new/src/Language/C/Data/Ident.hs 2011-01-06 15:31:57.632007216 -0600 -@@ -17,7 +17,7 @@ - module Language.C.Data.Ident ( - Ident(..), - SUERef(..), isAnonymousRef, -- mkIdent, builtinIdent, internalIdent, internalIdentAt, isInternalIdent, identToString, dumpIdent) -+ mkIdent, builtinIdent, internalIdent, internalIdentAt, isInternalIdent, identToString, dumpIdent, hashIdent) - where - - -- TODO (comment from manuel): -@@ -133,3 +133,6 @@ - -- | dump the identifier string and its positions for debugging purposes - dumpIdent :: Ident -> String - dumpIdent ide = identToString ide ++ " at " ++ show (nodeInfo ide) -+ -+hashIdent :: Ident -> Int -+hashIdent (Ident _ h _) = h -diff -Naur language-c-0.4/src/Language/C/Data/Position.hs language-c-0.4-new/src/Language/C/Data/Position.hs ---- language-c-0.4/src/Language/C/Data/Position.hs 2011-01-06 11:46:35.000000000 -0600 -+++ language-c-0.4-new/src/Language/C/Data/Position.hs 2011-01-06 15:31:57.634482017 -0600 -@@ -50,7 +50,7 @@ - show BuiltinPosition = "<builtin>" - show InternalPosition = "<internal>" - --{-# DEPRECATED posColumn "column number information is inaccurate in presence of macros - do not rely on it." #-} -+{-- DEPRECATED posColumn "column number information is inaccurate in presence of macros - do not rely on it." --} - - -- | @position absoluteOffset fileName lineNumber columnNumber@ initializes a @Position@ using the given arguments - position :: Int -> String -> Int -> Int -> Position -diff -Naur language-c-0.4/src/Language/C/Parser/Lexer.x language-c-0.4-new/src/Language/C/Parser/Lexer.x ---- language-c-0.4/src/Language/C/Parser/Lexer.x 2011-01-06 11:46:35.000000000 -0600 -+++ language-c-0.4-new/src/Language/C/Parser/Lexer.x 2011-01-06 15:31:57.636922062 -0600 -@@ -351,6 +351,13 @@ - idkwtok ('v' : 'o' : 'l' : 'a' : 't' : 'i' : 'l' : 'e' : []) = tok 8 CTokVolatile - idkwtok ('_' : '_' : 'v' : 'o' : 'l' : 'a' : 't' : 'i' : 'l' : 'e' : '_' : '_' : []) = tok 12 CTokVolatile - idkwtok ('w' : 'h' : 'i' : 'l' : 'e' : []) = tok 5 CTokWhile -+idkwtok ('_':'_':'b':'l':'o':'c':'k':'i':'n':'g':[]) = tok 10 CTokBlocking -+idkwtok ('_':'_':'r':'e':'m':'o':'t':'e':[]) = tok 10 CTokRemote -+idkwtok ('p':'w':'a':'i':'t':[]) = tok 5 CTokPWait -+idkwtok ('p':'b':'r':'a':'n':'c':'h':[]) = tok 7 CTokPBranch -+idkwtok ('p':'b':'r':'e':'a':'k':[]) = tok 6 CTokPBreak -+idkwtok ('p':'p':'r':'i':'v':'a':'t':'e':[]) = tok 6 CTokPPrivate -+idkwtok ('p':'s':'h':'a':'r':'e':'d':[]) = tok 6 CTokPShared - - idkwtok cs = \pos -> do - name <- getNewName -diff -Naur language-c-0.4/src/Language/C/Parser/Parser.y language-c-0.4-new/src/Language/C/Parser/Parser.y ---- language-c-0.4/src/Language/C/Parser/Parser.y 2011-01-11 11:33:19.331089585 -0600 -+++ language-c-0.4-new/src/Language/C/Parser/Parser.y 2011-01-11 11:44:45.593786776 -0600 -@@ -217,6 +217,8 @@ - typedef { CTokTypedef _ } - typeof { CTokTypeof _ } - "__thread" { CTokThread _ } -+"__blocking" { CTokBlocking _ } -+"__remote" { CTokRemote _ } - union { CTokUnion _ } - unsigned { CTokUnsigned _ } - void { CTokVoid _ } -@@ -236,6 +238,11 @@ - "__builtin_va_arg" { CTokGnuC GnuCVaArg _ } - "__builtin_offsetof" { CTokGnuC GnuCOffsetof _ } - "__builtin_types_compatible_p" { CTokGnuC GnuCTyCompat _ } -+pwait { CTokPWait _ } -+pbranch { CTokPBranch _ } -+pbreak { CTokPBreak _ } -+pshared { CTokPShared _ } -+pprivate { CTokPPrivate _ } - - %% - -@@ -359,6 +366,7 @@ - | iteration_statement { $1 } - | jump_statement { $1 } - | asm_statement {% withNodeInfo $1 (CAsm $1) } -+ | parallel_statement { $1 } - - - -- parse C labeled statement (C99 6.8.1) -@@ -473,6 +481,16 @@ - | for '(' enter_scope declaration expression_opt ';' expression_opt ')' statement leave_scope - {% withNodeInfo $1 $ CFor (Right $4) $5 $7 $9 } - -+-- parse Parallel C statement -+-- -+parallel_statement :: { CStat } -+parallel_statement -+ : pwait statement -+ {% withNodeInfo $1 $ CPWait $2 } -+ | pbranch statement -+ {% withNodeInfo $1 $ CPBranch $2 } -+ | pbreak -+ {% withNodeInfo $1 $ CPBreak } - - -- parse C jump statement (C99 6.8.6) - -- -@@ -550,7 +568,7 @@ - -- storage class and type qualifier - --------------------------------------------------------------------------------------------------------------- - attr :- __attribute__((..)) --storage_class :- typedef | extern | static | auto | register | __thread -+storage_class :- typedef | extern | static | auto | register | __thread | __blocking | pprivate | ppshared - type_qualifier :- const | volatile | restrict | inline - type_qualifier_list :- type_qualifier+ - -@@ -755,7 +773,7 @@ - - - -- A mixture of type qualifiers (const, volatile, restrict, inline) and storage class specifiers ---- (extern, static, auto, register, __thread), in any order, but containing at least one storage class specifier. -+-- (extern, static, auto, register, __thread, __blocking, pshared, pprivate), in any order, but containing at least one storage class specifier. - -- - -- declaration_qualifier_list :- <permute> type_qualifier* storage_class+ - -- -@@ -1188,6 +1206,10 @@ - | volatile {% withNodeInfo $1 $ CVolatQual } - | restrict {% withNodeInfo $1 $ CRestrQual } - | inline {% withNodeInfo $1 $ CInlineQual } -+ | "__blocking" {% withNodeInfo $1 $ CBlocking } -+ | "__remote" {% withNodeInfo $1 $ CRemote } -+ | pshared {% withNodeInfo $1 $ CPShared } -+ | pprivate {% withNodeInfo $1 $ CPPrivate } - - -- a list containing at least one type_qualifier (const, volatile, restrict, inline) - -- and additionally CAttrs -@@ -2161,13 +2183,17 @@ - withAsmNameAttrs (mAsmName, newAttrs) declr = setAsmName mAsmName (appendObjAttrsR newAttrs declr) - - appendDeclrAttrs :: [CAttr] -> CDeclrR -> CDeclrR --appendDeclrAttrs newAttrs (CDeclrR ident (Reversed []) asmname cattrs at) -- = CDeclrR ident empty asmname (cattrs ++ newAttrs) at - appendDeclrAttrs newAttrs (CDeclrR ident (Reversed (x:xs)) asmname cattrs at) -- = CDeclrR ident (Reversed (appendAttrs x : xs)) asmname cattrs at where -- appendAttrs (CPtrDeclr typeQuals at) = CPtrDeclr (typeQuals ++ map CAttrQual newAttrs) at -- appendAttrs (CArrDeclr typeQuals arraySize at) = CArrDeclr (typeQuals ++ map CAttrQual newAttrs) arraySize at -- appendAttrs (CFunDeclr parameters cattrs at) = CFunDeclr parameters (cattrs ++ newAttrs) at -+ | hasFunDeclr (x:xs) = CDeclrR ident (Reversed (appendAttrs x : xs)) asmname cattrs at -+ | otherwise = CDeclrR ident (Reversed (x:xs)) asmname (cattrs ++ newAttrs) at -+ where appendAttrs (CPtrDeclr typeQuals at) = CPtrDeclr (typeQuals ++ map CAttrQual newAttrs) at -+ appendAttrs (CArrDeclr typeQuals arraySize at) = CArrDeclr (typeQuals ++ map CAttrQual newAttrs) arraySize at -+ appendAttrs (CFunDeclr parameters cattrs at) = CFunDeclr parameters (cattrs ++ newAttrs) at -+ isFunDeclr (CFunDeclr _ _ _) = True -+ isFunDeclr _ = False -+ hasFunDeclr ds = any isFunDeclr ds -+appendDeclrAttrs newAttrs (CDeclrR ident derived asmname cattrs at) -+ = CDeclrR ident derived asmname (cattrs ++ newAttrs) at - - ptrDeclr :: CDeclrR -> [CTypeQual] -> NodeInfo -> CDeclrR - ptrDeclr (CDeclrR ident derivedDeclrs asmname cattrs dat) tyquals at -diff -Naur language-c-0.4/src/Language/C/Parser/Tokens.hs language-c-0.4-new/src/Language/C/Parser/Tokens.hs ---- language-c-0.4/src/Language/C/Parser/Tokens.hs 2011-01-06 11:46:35.000000000 -0600 -+++ language-c-0.4-new/src/Language/C/Parser/Tokens.hs 2011-01-06 15:31:57.643469288 -0600 -@@ -132,6 +132,13 @@ - | CTokTyIdent !PosLength !Ident -- `typedef-name' identifier - | CTokGnuC !GnuCTok !PosLength -- special GNU C tokens - | CTokEof -- end of file -+ | CTokBlocking !PosLength -- '__blocking' -+ | CTokRemote !PosLength -- '__remote' -+ | CTokPWait !PosLength -- `pwait' -+ | CTokPBranch !PosLength -- `pbranch' -+ | CTokPBreak !PosLength -- `pbreak' -+ | CTokPShared !PosLength -- `pshared' -+ | CTokPPrivate !PosLength -- `pprivate' - - -- special tokens used in GNU C extensions to ANSI C - -- -@@ -242,6 +249,13 @@ - posLenOfTok (CTokIdent pos _) = pos - posLenOfTok (CTokTyIdent pos _) = pos - posLenOfTok (CTokGnuC _ pos ) = pos -+posLenOfTok (CTokBlocking pos ) = pos -+posLenOfTok (CTokRemote pos ) = pos -+posLenOfTok (CTokPWait pos ) = pos -+posLenOfTok (CTokPBranch pos ) = pos -+posLenOfTok (CTokPBreak pos ) = pos -+posLenOfTok (CTokPPrivate pos ) = pos -+posLenOfTok (CTokPShared pos ) = pos - posLenOfTok CTokEof = error "tokenPos: Eof" - - instance Show CToken where -@@ -327,11 +341,18 @@ - showsPrec _ (CTokTypedef _ ) = showString "typedef" - showsPrec _ (CTokTypeof _ ) = showString "typeof" - showsPrec _ (CTokThread _ ) = showString "__thread" -+ showsPrec _ (CTokBlocking _ ) = showString "__blocking" -+ showsPrec _ (CTokRemote _ ) = showString "__remote" - showsPrec _ (CTokUnion _ ) = showString "union" - showsPrec _ (CTokUnsigned _ ) = showString "unsigned" - showsPrec _ (CTokVoid _ ) = showString "void" - showsPrec _ (CTokVolatile _ ) = showString "volatile" - showsPrec _ (CTokWhile _ ) = showString "while" -+ showsPrec _ (CTokPWait _ ) = showString "pwait" -+ showsPrec _ (CTokPBranch _ ) = showString "pbranch" -+ showsPrec _ (CTokPBreak _ ) = showString "pbreak" -+ showsPrec _ (CTokPPrivate _ ) = showString "pprivate" -+ showsPrec _ (CTokPShared _ ) = showString "pshared" - showsPrec _ (CTokCLit _ c) = shows c - showsPrec _ (CTokILit _ i) = shows i - showsPrec _ (CTokFLit _ f) = shows f -diff -Naur language-c-0.4/src/Language/C/Pretty.hs language-c-0.4-new/src/Language/C/Pretty.hs ---- language-c-0.4/src/Language/C/Pretty.hs 2011-01-06 11:46:35.000000000 -0600 -+++ language-c-0.4-new/src/Language/C/Pretty.hs 2011-01-06 15:31:57.646019992 -0600 -@@ -154,7 +154,11 @@ - pretty (CReturn Nothing _) = ii $ text "return" <> semi - pretty (CReturn (Just e) _) = ii $ text "return" <+> pretty e <> semi - pretty (CAsm asmStmt _) = pretty asmStmt -- prettyPrec p (CCompound localLabels bis _) = -+ pretty (CPWait stmt _) = -+ ii $ text "pwait" $+$ prettyPrec (-1) stmt -+ pretty (CPBranch stmt _) = ii $ text "pbranch" $+$ prettyPrec (-1) stmt -+ pretty (CPBreak _) = ii $ text "pbreak" <> semi -+ prettyPrec p (CCompound localLabels bis ni) = - let inner = text "{" $+$ mlistP ppLblDecls localLabels $+$ vcat (map pretty bis) $$ text "}" - in if p == -1 then inner else ii inner - where ppLblDecls = vcat . map (\l -> text "__label__" <+> identP l <+> semi) -@@ -253,6 +257,10 @@ - pretty (CRestrQual _) = text "__restrict" - pretty (CInlineQual _) = text "inline" - pretty (CAttrQual a) = attrlistP [a] -+ pretty (CBlocking _) = text "__blocking" -+ pretty (CRemote _) = text "__remote" -+ pretty (CPShared _) = text "pshared" -+ pretty (CPPrivate _) = text "pprivate" - - instance Pretty CStructUnion where - pretty (CStruct tag ident Nothing cattrs _) = pretty tag <+> attrlistP cattrs <+> maybeP identP ident -@@ -375,8 +383,8 @@ - <+> maybeP pretty expr2 <+> text ":" <+> prettyPrec 4 expr3 - prettyPrec p (CBinary op expr1 expr2 _) = - let prec = binPrec op -- in parenPrec p prec $ prettyPrec prec expr1 -- <+> pretty op <+> prettyPrec (prec + 1) expr2 -+ in parenPrec p prec $ text "(" <+> prettyPrec prec expr1 -+ <+> pretty op <+> prettyPrec (prec + 1) expr2 <+> text ")" - prettyPrec p (CCast decl expr _) = - parenPrec p 25 $ text "(" <> pretty decl <> text ")" - <+> prettyPrec 25 expr -diff -Naur language-c-0.4/src/Language/C/Syntax/AST.hs language-c-0.4-new/src/Language/C/Syntax/AST.hs ---- language-c-0.4/src/Language/C/Syntax/AST.hs 2011-01-06 11:46:35.000000000 -0600 -+++ language-c-0.4-new/src/Language/C/Syntax/AST.hs 2011-01-06 15:31:57.648960933 -0600 -@@ -276,6 +276,9 @@ - | CReturn (Maybe (CExpression a)) a - -- | assembly statement - | CAsm CAsmStmt a -+ | CPWait CStat a -+ | CPBranch CStat a -+ | CPBreak a - deriving (Data,Typeable {-! CNode !-}) - - -- | GNU Assembler statement -@@ -399,6 +402,10 @@ - | CRestrQual a - | CInlineQual a - | CAttrQual (CAttribute a) -+ | CBlocking a -- ^ blocking function -+ | CRemote a -- ^ remote function -+ | CPShared a -- ^ shared variable across pbranch blocks -+ | CPPrivate a -- ^ private variable across pbranch blocks - deriving (Data,Typeable {-! CNode !-}) - - -- | C structure or union specifiers (K&R A8.3, C99 6.7.2.1) -@@ -663,6 +670,9 @@ - nodeInfo (CBreak d) = nodeInfo d - nodeInfo (CReturn _ t) = nodeInfo t - nodeInfo (CAsm _ t) = nodeInfo t -+ nodeInfo (CPWait _ t) = nodeInfo t -+ nodeInfo (CPBranch _ t) = nodeInfo t -+ nodeInfo (CPBreak t) = nodeInfo t - instance CNode t1 => Pos (CStatement t1) - where posOf x = posOfNode (nodeInfo x) - -@@ -726,6 +736,10 @@ - nodeInfo (CRestrQual d) = nodeInfo d - nodeInfo (CInlineQual d) = nodeInfo d - nodeInfo (CAttrQual d) = nodeInfo d -+ nodeInfo (CBlocking d) = nodeInfo d -+ nodeInfo (CRemote d) = nodeInfo d -+ nodeInfo (CPPrivate d) = nodeInfo d -+ nodeInfo (CPShared d) = nodeInfo d - instance CNode t1 => Pos (CTypeQualifier t1) - where posOf x = posOfNode (nodeInfo x) - diff --git a/code/maint/hs/setup-aesop b/code/maint/hs/setup-aesop deleted file mode 100755 index d384a9d..0000000 --- a/code/maint/hs/setup-aesop +++ /dev/null @@ -1,47 +0,0 @@ -#!/bin/bash - -# SETUP-AESOP -# Downloads language-c -# Patches it with Aesop modifications, making package language-aesop -# Installs it with cabal - -# Absolute path to maint/hs: -MAINT_HS=$( cd $( dirname $0 ) ; /bin/pwd ) - -VERSION=0.4.2 - -pushd ${MAINT_HS} -wget --connect-timeout=2 --read-timeout=2 --tries=1 http://hackage.haskell.org/packages/archive/language-c/${VERSION}/language-… -popd - -# Put happy and alex in the path -PATH=$HOME/.cabal/bin:$PATH - -tar xfz ${MAINT_HS}/language-c-${VERSION}.tar.gz >/dev/null -if [[ $? != 0 ]] -then - echo "Untar of language-c-${VERSION}.tar.gz failed!" - exit 1 -fi - -echo "Patching language-c-${VERSION} with Aesop extensions..." - -cd language-c-${VERSION} -patch -p 1 < ${MAINT_HS}/language-c-${VERSION}.patch > /dev/null -if [[ $? != 0 ]] -then - echo "Patch of language-c failed!" - exit 1 -fi - -echo "Building language-aesop-${VERSION} package..." -cabal configure -cabal build -echo "Installing language-aesop-${VERSION} package..." -cabal install - -# rm -r language-c-${VERSION} - -# Local Variables: -# sh-basic-offset: 4 -# End: diff --git a/code/maint/hs/setup-cabal-local b/code/maint/hs/setup-cabal-local deleted file mode 100755 index 60abc08..0000000 --- a/code/maint/hs/setup-cabal-local +++ /dev/null @@ -1,11 +0,0 @@ -#!/bin/sh - -# SETUP-CABAL-LOCAL -# Installs cabal locally -# Use apt-get install cabal-install to install globally - -wget http://www.haskell.org/cabal/release/cabal-install-0.14.0/cabal-install-0.1… -tar xvfz cabal-install-0.14.0.tar.gz -cd cabal-install-0.14.0 -sh ./bootstrap.sh - diff --git a/code/maint/hs/setup-hs-local b/code/maint/hs/setup-hs-local deleted file mode 100755 index 5367231..0000000 --- a/code/maint/hs/setup-hs-local +++ /dev/null @@ -1,12 +0,0 @@ -#!/bin/sh - -# SETUP-HS-LOCAL -# Uses cabal to install Haskell packages required for Aesop - -cabal update - -cabal install alex -cabal install happy -cabal install regex-posix -cabal install regex-pcre -cabal install mtl diff --git a/code/src/aesop/ae-remote-parser.h b/code/src/aesop/ae-remote-parser.h deleted file mode 100644 index 887c6dd..0000000 --- a/code/src/aesop/ae-remote-parser.h +++ /dev/null @@ -1,566 +0,0 @@ -#ifndef __AE_REMOTE_PARSER_H__ -#define __AE_REMOTE_PARSER_H__ - -#include <triton-list.h> -#include <aesop/hints.h> - -#include "src/common/triton-types.h" - -#include "src/common/triton-error.h" -#include "src/common/triton-log.h" -#include "src/remote/remote.hae" -#include "src/remote/encoding.h" -#include "src/common/triton-debug.h" - -extern triton_debug_mask_t encoding_dbg_mask; - -/* TODO: the following typedefs should ideally be defined in - * test-remote-list.aer and sc.haer, respectively. That causes a - * compilation error because only headers known to the Aesop parser - * are available at compilation time. - */ -typedef triton_list_t* triton_state_table_list_t; -typedef triton_list_t* triton_state_data_list_t; - -/** - * The remote parser generates encoding/decoding functions for every type (struct, typedef, etc.) with - * the __remote specifier. This header allows callouts to be defined, so that compile time configuration - * changes (made through CPP #defines) can be made to the internals of the encoding/decoding functions. - * The header defines a set of macros that get invoked by the parser to generate declarations or statements that - * are placed in the encoding/decoding functions. To get an idea of how this works, the parser - * generates an encoding function that looks like this: - * - * static inline triton_ret_t aer_encode_<type>(ae_buffer_t buf, const char *field_name, void *vx) - * { - * AE_MK_ENCODE_DECLS(<type>); - * AE_MK_ENCODE_STMTS_START(<type>); - * - * AE_MK_ENCODE_TYPE(<subtype1>, <subfield1>); - * AE_MK_ENCODE_TYPE(<subtype2>, <subfield2>); - * ... - * - * AE_MK_ENCODE_STMTS_END(<type>); - * } - * - * For fields that are pointers, the AE_MK_ENCODE_TYPE_PTR macro is called instead. - * - * The macros below are defined generate the appropriate code for the above function. - */ - -#define AER_MK_ENCODE_TYPE(__type__, __field__, __ptr__) \ -{ \ - ret = aer_encode_##__type__(buf, #__field__, __ptr__ __aer_x->__field__); \ - if(ret != TRITON_SUCCESS) \ - { \ - return ret; \ - } \ -} - -#define AER_MK_ENCODE_DECLS(__type__) \ - triton_ret_t ret = TRITON_SUCCESS; \ - __type__ __aer_x; - -#define AER_MK_ENCODE_STMTS_START(__type__, __canon_type__) \ -{ \ - __aer_x = (__type__)vx; \ - triton_debug(encoding_dbg_mask, "encoding:" #__canon_type__ ":%s\t cptr=%p\n", varname, triton_buffer_cptr(buf)); \ -} - -#define AER_MK_ENCODE_STMTS_END() \ -{ \ - return ret; \ -} - -/** - * Simlarly, the decode function looks like this: - * - * triton_ret_t aer_decode_<type>(ae_buffer_t buf, const char *field_name, void *vx) - * { - * AER_MK_DECODE_DECLS(<type>); - * AER_MK_DECODE_STMTS_START(<type>); - * - * AER_MK_DECODE_TYPE(<subtype1>, <subfield1>); - * AER_MK_DECODE_TYPE(<subtype2>, <subfield2>); - * ... - * - * AER_MK_DECODE_STMTS_END(<type>); - * } - * - * As before, for fields that are pointers, the AER_MK_DECODE_TYPE_PTR macro is called instead. - */ - -#define AER_MK_DECODE_TYPE(__type__, __field__, __ptr__) \ -{ \ - ret = aer_decode_##__type__(buf, NULL, __ptr__ __aer_x->__field__); \ - if(ret != TRITON_SUCCESS) \ - { \ - return ret; \ - } \ -} - -#define AER_MK_DECODE_DECLS(__type__) \ - triton_ret_t ret = TRITON_SUCCESS; \ - __type__ __aer_x; - -#define AER_MK_DECODE_STMTS_START(__type__, __canon_type__) \ -{ \ - __aer_x = (__type__)vx; \ - triton_debug(encoding_dbg_mask, "decoding:" #__canon_type__ ":%s\t cptr=%p\n", (varname ? *varname : ""), triton_buffer_cptr(buf)); \ -} - -#define AER_MK_DECODE_STMTS_END() \ -{ \ - return ret; \ -} - -/** - * Simlarly, the size encode_size function looks like this: - * - * uint64_t aer_encode_size_<type>(const char *field_name, void *vx) - * { - * AER_MK_SIZE_DECLS(<type>); - * AER_MK_SIZE_STMTS_START(<type>); - * - * AER_MK_SIZE_TYPE(<subtype1>, <subfield1>); - * AER_MK_SIZE_TYPE(<subtype2>, <subfield2>); - * ... - * - * AER_MK_SIZE_STMTS_END(<type>); - * } - * - * As before, for fields that are pointers, the AER_MK_SIZE_TYPE_PTR macro is called instead. - */ - -#define AER_MK_SIZE_TYPE(__type__, __field__, __ptr__) \ -{ \ - size += aer_encode_size_##__type__ (#__field__, __ptr__ __aer_x->__field__); \ -} - -#define AER_MK_SIZE_DECLS(__type__) \ - uint64_t size; \ - __type__ __aer_x; - -#define AER_MK_SIZE_STMTS_START(__type__) \ -{ \ - size = 0; \ - __aer_x = (__type__)vx; \ -} - -#define AER_MK_SIZE_STMTS_END() \ -{ \ - return size; \ -} - -/** - * Only called on pointer fields to pre-initialize - */ -#define AER_MK_INIT_PTR_NULL(__type__, __field__) \ -{ \ - __aer_x->__field__ = NULL; \ -} - -#define AER_MK_INIT_NULL_TYPE(__basetype__, __type__, __field__, __ptr__) \ -{ \ - ret = aer_init_null_##__type__(__ptr__ __aer_x->__field__); \ - if(ret != TRITON_SUCCESS) \ - { \ - aer_destroy_##__basetype__(__aer_x); \ - return ret; \ - } \ -} - -#define AER_MK_PTR_INIT_TYPE(__basetype__, __type__, __field__, __ptr__) \ -{ \ - __aer_x->__field__ = __field__; \ -} - -#define AER_MK_INIT_TYPE(__basetype__, __type__, __field__, __ptr__) \ -{ \ - ret = aer_copy_##__type__(__ptr__ __aer_x->__field__, __field__); \ - if(ret != TRITON_SUCCESS) \ - { \ - aer_destroy_##__basetype__(__aer_x); \ - return ret; \ - } \ -} - -#define AER_MK_INIT_DECLS(__type__) \ - triton_ret_t ret = TRITON_SUCCESS; \ - __type__ __aer_x; - -#define AER_MK_INIT_STMTS_START(__type__) \ -{ \ - __aer_x = (__type__)vx; \ -} - -#define AER_MK_INIT_STMTS_END() \ -{ \ - return ret; \ -} - -#define AER_MK_PTR_COPY_TYPE(__basetype__, __type__, __field__, __ptr__) \ -{ \ - __aer_x->__field__ = v->__field__; \ -} - -#define AER_MK_COPY_TYPE(__basetype__, __type__, __field__, __ptr__) \ -{ \ - ret = aer_copy_##__type__(__ptr__ __aer_x->__field__, __ptr__ v->__field__); \ - if(ret != TRITON_SUCCESS) \ - { \ - aer_destroy_##__basetype__(__aer_x); \ - return ret; \ - } \ -} - -#define AER_MK_COPY_DECLS(__type__) \ - triton_ret_t ret = TRITON_SUCCESS; \ - __type__ __aer_x; \ - __type__ v; - -#define AER_MK_COPY_STMTS_START(__type__) \ -{ \ - __aer_x = (__type__)vx; \ - v = (__type__)vv; \ -} - -#define AER_MK_COPY_STMTS_END() \ -{ \ - return ret; \ -} - -#define AER_MK_DESTROY_TYPE(__basetype__, __type__, __field__, __ptr__) \ -{ \ - aer_destroy_##__type__(__ptr__ __aer_x->__field__); \ -} - -#define AER_MK_DESTROY_DECLS(__type__) \ - __type__ __aer_x; - -#define AER_MK_DESTROY_STMTS_START(__type__) \ -{ \ - __aer_x = (__type__)vx; \ -} - -#define AER_MK_DESTROY_STMTS_END() - -#define AER_MK_STUB_DECL(__ret__, __fname__, params...) \ - __blocking __ret__ remote_##__fname__(aer_remote_ctx_t ctx, triton_addr_t id, ##params); - -#define AER_MK_STUB_DECLS(__fname__) \ - aer_message_t send_message; \ - aer_message_t recv_message; \ - triton_ret_t ret, sret; \ - uint64_t insize; \ - uint64_t op; \ - int s; - -#define AER_MK_STUB_BLOCK(__fname__, __intype__, __inname__, __outtype__, __outname__, __ptr__) \ -{ \ - s = triton_addr_is_self(id); \ - if(s) \ - { \ - int ae_ret; \ - ae_ret = ae_hints_put(&__ae_ctl->gen.hints, "triton.remote.context", sizeof(ctx), &ctx, 1); \ - if(ae_ret != AE_SUCCESS) triton_log_error(triton_log_default, TRITON_ERR_UNKNOWN, "Failed to store remote context in hints\n"); \ - ae_ret = ae_hints_put(&__ae_ctl->gen.hints, "triton.remote.from", sizeof(id), &id, 1); \ - if(ae_ret != AE_SUCCESS) triton_log_error(triton_log_default, TRITON_ERR_UNKNOWN, "Failed to store remote address in hints\n"); \ - return __fname__(__inname__, __outname__); \ - } \ - insize = aer_encode_size_##__intype__(#__inname__, __ptr__ __inname__); \ - ret = aer_message_init(ctx, &send_message, AER_MESSAGE_REQUEST, insize); \ - if(ret != TRITON_SUCCESS) \ - { \ - return ret; \ - } \ - op = __get_op_id_##__fname__(); \ - ret = aer_message_attr_set(ctx, &send_message, "aesop.remote.operation", &op); \ - if(ret != TRITON_SUCCESS) \ - { \ - return ret; \ - } \ - triton_debug(encoding_dbg_mask, "---------- encoding request header: " #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&send_message.buffer)); \ - ret = aer_message_encode(ctx, &send_message); \ - triton_debug(encoding_dbg_mask, "---------- encoding request header completed: " #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&send_message.buffer)); \ - if(ret != TRITON_SUCCESS) \ - { \ - return ret; \ - } \ - triton_debug(encoding_dbg_mask, "---------- encoding request: " #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&send_message.buffer)); \ - ret = aer_encode_##__intype__(&(send_message.buffer), #__inname__, __ptr__ __inname__); \ - if(ret != TRITON_SUCCESS) \ - { \ - triton_debug(encoding_dbg_mask, "---------- encoding request completed: " #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&send_message.buffer)); \ - aer_message_destroy(ctx, &send_message); \ - return ret; \ - } \ - ret = aer_encoding_finish(&(send_message.buffer)); \ - triton_debug(encoding_dbg_mask, "---------- encoding request completed: " #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&send_message.buffer)); \ - if(ret != TRITON_SUCCESS) \ - { \ - aer_message_destroy(ctx, &send_message); \ - return ret; \ - } \ - ret = aer_message_init(ctx, &recv_message, AER_MESSAGE_RESPONSE, AE_REMOTE_MAX_RESPONSE_SIZE); \ - if(ret != TRITON_SUCCESS) \ - { \ - aer_message_destroy(ctx, &send_message); \ - return ret; \ - } \ - ret = aer_message_sendrecv(ctx, id, &send_message, &recv_message); \ - if(ret != TRITON_SUCCESS) \ - { \ - aer_message_destroy(ctx, &send_message); \ - aer_message_destroy(ctx, &recv_message); \ - return ret; \ - } \ - triton_debug(encoding_dbg_mask, "---------- decoding response header:" #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&recv_message.buffer)); \ - ret = aer_message_decode(ctx, &recv_message); \ - triton_debug(encoding_dbg_mask, "---------- decoding response header completed:" #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&recv_message.buffer)); \ - if(ret != TRITON_SUCCESS) \ - { \ - aer_message_destroy(ctx, &send_message); \ - aer_message_destroy(ctx, &recv_message); \ - return ret; \ - } \ - triton_debug(encoding_dbg_mask, "---------- decoding response: " #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&recv_message.buffer)); \ - triton_debug(encoding_dbg_mask, "decoding:return cptr=%p\n", triton_buffer_cptr(&recv_message.buffer)); \ - ret = aer_decode_triton_ret_t(&(recv_message.buffer), NULL, &sret); \ - if(ret != TRITON_SUCCESS) \ - { \ - triton_debug(encoding_dbg_mask, "---------- decoding response completed: " #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&recv_message.buffer)); \ - aer_message_destroy(ctx, &send_message); \ - aer_message_destroy(ctx, &recv_message); \ - return ret; \ - } \ - if(sret == TRITON_SUCCESS) \ - { \ - triton_debug(encoding_dbg_mask, "decoding:" #__outtype__ ":" #__outname__ " cptr=%p\n", \ - triton_buffer_cptr(&recv_message.buffer)); \ - ret = aer_decode_##__outtype__(&(recv_message.buffer), NULL, __outname__); \ - triton_debug(encoding_dbg_mask, "---------- decoding response completed: " #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&recv_message.buffer)); \ - aer_message_destroy(ctx, &send_message); \ - aer_message_destroy(ctx, &recv_message); \ - if(ret != TRITON_SUCCESS) \ - { \ - return ret; \ - } \ - } \ - else \ - { \ - triton_debug(encoding_dbg_mask, "---------- decoding response completed: cptr=%p ----------\n", \ - triton_buffer_cptr(&recv_message.buffer)); \ - aer_message_destroy(ctx, &send_message); \ - aer_message_destroy(ctx, &recv_message); \ - } \ - return sret; \ -} - -#define AER_MK_SERVICE_DECL(__fname__) \ - __blocking triton_ret_t __service_##__fname__( \ - aer_remote_ctx_t ctx, aer_message_t *send_message, aer_message_t *recv_message, triton_ret_t *service_error); \ - triton_ret_t __service_init_##__fname__( \ - aer_remote_ctx_t ctx, aer_message_t *in_message, aer_message_t *out_message); \ - void __service_destroy_##__fname__( \ - aer_remote_ctx_t ctx, triton_ret_t service_return, aer_message_t *in_message, aer_message_t *out_message); \ - struct aer_service_ctx __service_##__fname__##_ctx; - - -#define AER_MK_SERVICE_FNDEF(__fname__, \ - __intype__, \ - __incanontype__, \ - __inname__, \ - __outtype__, \ - __outcanontype__, \ - __outname__, \ - __ptr__) \ -__blocking triton_ret_t __service_##__fname__( \ - aer_remote_ctx_t ctx, aer_message_t *in_message, aer_message_t *out_message, triton_ret_t *service_error) \ -{ \ - triton_ret_t ret, sret; \ - uint32_t outsize, retsize; \ - triton_debug(encoding_dbg_mask, "---------- decoding request: " #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&in_message->buffer)); \ - ret = aer_decode_##__incanontype__(&(in_message->buffer), NULL, in_message->param); \ - triton_debug(encoding_dbg_mask, "---------- decoding request completed: " #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&in_message->buffer)); \ - if(ret != TRITON_SUCCESS) \ - { \ - return ret; \ - } \ - sret = __fname__(__ptr__ (__intype__ *)in_message->param, (__outtype__ *)out_message->param); \ - *service_error = sret; \ - retsize = aer_encode_size_triton_ret_t("error", &sret); \ - outsize = 0; \ - if(sret == TRITON_SUCCESS) \ - { \ - outsize = aer_encode_size_##__outcanontype__(#__outname__, out_message->param); \ - } \ - ret = aer_message_init(ctx, out_message, AER_MESSAGE_RESPONSE, retsize+outsize); \ - if(ret != TRITON_SUCCESS) \ - { \ - return ret; \ - } \ - triton_debug(encoding_dbg_mask, "---------- encoding response header: " #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&out_message->buffer)); \ - ret = aer_message_encode(ctx, out_message); \ - triton_debug(encoding_dbg_mask, "---------- encoding response header completed: " #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&out_message->buffer)); \ - if(ret != TRITON_SUCCESS) \ - { \ - aer_message_destroy(ctx, out_message); \ - return ret; \ - } \ - triton_debug(encoding_dbg_mask, "---------- encoding response: " #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&out_message->buffer)); \ - ret = aer_encode_triton_ret_t(&(out_message->buffer), "error", &sret); \ - if(ret != TRITON_SUCCESS) \ - { \ - aer_message_destroy(ctx, out_message); \ - return ret; \ - } \ - if(sret == TRITON_SUCCESS) \ - { \ - ret = aer_encode_##__outcanontype__(&(out_message->buffer), #__outname__, out_message->param); \ - if(ret != TRITON_SUCCESS) \ - { \ - triton_debug(encoding_dbg_mask, "---------- encoding response completed: " #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&out_message->buffer)); \ - aer_message_destroy(ctx, out_message); \ - return ret; \ - } \ - ret = aer_encoding_finish(&(out_message->buffer)); \ - triton_debug(encoding_dbg_mask, "---------- encoding request completed: " #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&out_message->buffer)); \ - if(ret != TRITON_SUCCESS) \ - { \ - aer_message_destroy(ctx, out_message); \ - return ret; \ - } \ - } \ - else \ - { \ - ret = aer_encoding_finish(&(out_message->buffer)); \ - triton_debug(encoding_dbg_mask, "---------- encoding request completed: " #__fname__ " cptr=%p ----------\n", \ - triton_buffer_cptr(&out_message->buffer)); \ - if(ret != TRITON_SUCCESS) \ - { \ - aer_message_destroy(ctx, out_message); \ - return ret; \ - } \ - } \ - return ret; \ -} - -#define AER_MK_SERVICE_INIT(__fname__, \ - __intype__, \ - __incanontype__, \ - __inname__, \ - __outtype__, \ - __outcanontype__, \ - __outname__, \ - __ptr__) \ -triton_ret_t __service_init_##__fname__( \ - aer_remote_ctx_t ctx, aer_message_t *in_message, aer_message_t *out_message) \ -{ \ - assert(in_message && out_message); \ - in_message->param = malloc(sizeof(__intype__)); \ - if(!in_message->param) \ - { \ - return TRITON_ERR_NOMEM; \ - } \ - out_message->param = malloc(sizeof(__outtype__)); \ - if(!out_message->param) \ - { \ - free(out_message->param); \ - out_message->param = NULL; \ - return TRITON_ERR_NOMEM; \ - } \ - return TRITON_SUCCESS; \ -} - -#define AER_MK_SERVICE_DESTROY(__fname__, \ - __intype__, \ - __incanontype__, \ - __inname__, \ - __outtype__, \ - __outcanontype__, \ - __outname__, \ - __ptr__) \ -void __service_destroy_##__fname__( \ - aer_remote_ctx_t ctx, triton_ret_t service_return, aer_message_t *in_message, aer_message_t *out_message) \ -{ \ - assert(in_message && out_message); \ - if(service_return == TRITON_SUCCESS) \ - { \ - aer_destroy_##__incanontype__(in_message->param); \ - aer_destroy_##__outcanontype__(out_message->param); \ - } \ - free(in_message->param); \ - in_message->param = NULL; \ - free(out_message->param); \ - out_message->param = NULL; \ -} - -#define AER_MK_SERVICE_CTX(__fname__) \ - struct aer_service_ctx __service_##__fname__##_ctx = \ - { .service_init = __service_init_##__fname__, \ - .service_destroy = __service_destroy_##__fname__, \ - .invoke_fn = __service_##__fname__ \ - }; - -#define AER_MK_OP_STMTS(__fname__) \ -{ \ - if(__op_id_##__fname__) \ - { \ - return __op_id_##__fname__; \ - } \ - __op_id_##__fname__ = aer_service_unique_id(#__fname__); \ - return __op_id_##__fname__; \ -} - -#define AER_MK_REG_DECLS(__service_name__) \ - triton_ret_t ret; - -#define AER_MK_REG_BLOCK(__fname__) \ -{ \ - ret = aer_service_register(__get_op_id_##__fname__(), #__fname__, &__service_##__fname__##_ctx); \ - if(ret != TRITON_SUCCESS) return ret; \ -} - -#define AER_MK_REG_START(__service_name__) \ - ret = TRITON_SUCCESS; - -#define AER_MK_REG_END(__service_name__) \ - return ret; - -#define AER_MK_STATIC_OP_DECL(__fname__) \ - static uint64_t __op_id_##__fname__ = 0; - -#define AER_MK_REG_CTOR_FNDEF(__service_name__) \ -static __attribute__((constructor)) void aer_remote_##__service_name__##_init_register(void) \ -{ \ -} - -#define AER_MK_STRUCT_DECLS(__name__, params...) \ - triton_ret_t aer_copy_##__name__(void * vx, void *vv); \ - void aer_destroy_##__name__(void * vx); \ - triton_ret_t aer_init_null_##__name__(void *vx); \ - uint64_t aer_encode_size_##__name__(const char *varname, void *vx); \ - triton_ret_t aer_encode_##__name__(triton_buffer_t *buf, const char *varname, void *vx); \ - triton_ret_t aer_decode_##__name__(triton_buffer_t *buf, char **varname, void *vx); - -#define AER_MK_STRUCT_INIT_DECL(__name__, params...) \ - triton_ret_t aer_init_##__name__(void *vx, ##params); - -#endif diff --git a/code/src/aesop/parser/BlockingContext.lhs b/code/src/aesop/parser/BlockingContext.lhs deleted file mode 100644 index 5a9b504..0000000 --- a/code/src/aesop/parser/BlockingContext.lhs +++ /dev/null @@ -1,65 +0,0 @@ -> module BlockingContext where -> import Language.C -> import Language.C.System.GCC -- preprocessor used -> import System.Environment -> import Data.Typeable -> import Data.Maybe -> import Data.Either -> import Data.HashTable -> import Control.Exception -> import Control.Monad.State -> import Data.List -> import Debug.Trace -> import CParse -> import CUtils -> import Data.Generics -> import Data.Generics.Schemes -> import Data.Tree - -> data BlockingContext = FunContext { funDef :: CFunDef } - -> | PWaitContext { pwaitStmt :: CStat , -> parent :: BlockingContext } - -> | PBranchContext { pbranchStmt :: CStat , -> parent :: BlockingContext } - -> isFunContext :: BlockingContext -> Bool -> isFunContext (FunContext _) = True -> isFunContext _ = False - -> isPWaitContext :: BlockingContext -> Bool -> isPWaitContext (PWaitContext _ _) = True -> isPWaitContext _ = False - -> isPBranchContext :: BlockingContext -> Bool -> isPBranchContext (PBranchContext _ _) = True -> isPBranchContext _ = False - -> parentIsPWait :: BlockingContext -> Bool -> parentIsPWait (PBranchContext _ parent) -> | isPWaitContext parent = True -> | otherwise = False -> parentIsPWait _ = False - -> getParent :: BlockingContext -> BlockingContext -> getParent b@(FunContext _) = b -> getParent b = getParent $ parent b - -> getParentName :: BlockingContext -> String -> getParentName (FunContext funDef) = getFunDefName funDef -> getParentName b = getParentName $ parent b - -> getPWaitSharedDecls :: CStat -> [CDecl] -> getPWaitSharedDecls (CPWait (CCompound _ bitems _) _) = -> filterDecls (\_ -> True) (not . isPrivateSpec) removeSharedSpecFromDecl $ findFuncDecls bitems -> getPWaitSharedDecls _ = [] - -> getPWaitPrivateDecls :: CStat -> [CDecl] -> getPWaitPrivateDecls (CPWait (CCompound _ bitems _) _) = -> filterDecls isPrivateSpec (not . isSharedSpec) removePrivateSpecFromDecl $ findFuncDecls bitems - -> getPBranchDecls :: CStat -> [CDecl] -> getPBranchDecls (CPBranch (CCompound _ bitems _) _) = -> findFuncDecls bitems -> getPBranchDecls _ = [] diff --git a/code/src/aesop/parser/CGen.lhs b/code/src/aesop/parser/CGen.lhs deleted file mode 100644 index b360339..0000000 --- a/code/src/aesop/parser/CGen.lhs +++ /dev/null @@ -1,787 +0,0 @@ -> module CGen where -> import Language.C -> import Language.C.System.GCC -- preprocessor used -> import Language.C.System.Preprocess -> import Language.C.Data.Position -> import Language.C.Data.Name -> import Language.C.Data.InputStream -> import System.Environment -> import Data.Typeable -> import Data.Maybe -> import Data.Either -> import Data.HashTable -> import Data.Generics -> import Data.Generics.Schemes -> import qualified Data.String.Utils as DSU -> import Control.Monad.State -> import qualified Data.ByteString -> import qualified Data.ByteString.Char8 -> import Control.Exception -> import Data.List -> import CParse -> import CUtils -> import System.IO -> import System.IO.Unsafe -> import System.Exit -> import System.Directory -> import System.Process -> import Debug.Trace -> import System.Time - -> tmpFile :: String -> IO FilePath -> tmpFile s = do -> (fp, h) <- openTempFile "/tmp" "__aesop_cgen_tmp.c" -> hPutStrLn h s -> hClose h -> return fp - -> newTmpFile :: IO FilePath -> newTmpFile = do -> (fp, h) <- openTempFile "/tmp" "__aesop_cgen_tmp.c" -> hClose h -> return fp - -> tmpHandle :: IO (FilePath, Handle) -> tmpHandle = do -> (fp, h) <- openTempFile "/tmp" "__aesop_cgen_tmp.c" -> return (fp, h) - -> getTimeDiff :: ClockTime -> ClockTime -> Float -> getTimeDiff (TOD c1s c1p) (TOD c2s c2p) = -> let u1 = (fromInteger c1s) * (1e6 :: Float) + (fromInteger c1p) * (1e-6 :: Float) -> u2 = (fromInteger c2s) * (1e6 :: Float) + (fromInteger c2p) * (1e-6 :: Float) -> in u2 - u1 - - -> myCPP :: String -> Maybe FilePath -> [String] -> (Either String FilePath) -> IO FilePath -> myCPP compiler incheader options input = do -> arp_debug ("myCPP:") -> inputFile <- either (\s -> tmpFile s) (\fp -> return fp) input -> outFile <- newTmpFile -> arp_debug ("myCPP: input: " ++ inputFile ++ " output: " ++ outFile) -> let mkIncHeader = if isJust incheader then ["-include", fromJust incheader] else [] -> let opts = ["-E"] ++ mkIncHeader ++ options ++ ["-o", outFile] ++ [inputFile] -> (errorFP, errorH) <- tmpHandle -> -- st <- getClockTime -> arp_debug ("myCPP: running: " ++ compiler ++ " " ++ (DSU.join " " opts)) -> inputContents <- (readFile inputFile) -> arp_debug ("myCPP: INPUT: \n--\n" ++ inputContents ++ "--\n") -> pid <- runProcess compiler opts Nothing Nothing Nothing Nothing (Just errorH) -> ec <- waitForProcess pid -> -- et <- getClockTime -> -- let td = getTimeDiff st et -> -- putStrLn $ (show td) ++ "us" -> -- either (\_ -> removeFile inputFile) (\_ -> return ()) input -> case ec of -> ExitSuccess -> do -> arp_debug "myCPP: OK\n" -> outputContents <- (readFile outFile) -> -- arp_debug ("myCPP: OUTPUT: \n--\n" ++ outputContents ++ "--\n") -> hClose errorH -> -- removeFile errorFP -> return outFile -> ExitFailure e -> do -> isOpen <- hIsOpen errorH -> errorH <- if not isOpen then openFile errorFP ReadMode else return errorH -> errs <- hGetContents errorH -> -- removeFile errorFP -> trace ("Preprocessor failed with return code: " ++ (show e) ++ "\n" ++ errs ++ "\n\n") $ assert False $ return "dummy_file.c" -> hClose errorH -> return "dummy_file.c" - -> type ReturnType = (CTypeSpec, [CDerivedDeclr]) - -> returnToString :: ReturnType -> String -> returnToString (t, ds) = (show $ pretty t) ++ (join (map derivedToString ds)) - -> derivedToString :: CDerivedDeclr -> String -> derivedToString (CPtrDeclr _ _) = " * " -> derivedToString (CFunDeclr (Right ([], _)) _ _) = "" -> derivedToString (CFunDeclr (Right (params, _)) _ _) = foldl1 (++) $ map (show . pretty) params - -Construct a new node info: - -> newIdent :: String -> NodeInfo -> Ident -> newIdent name ni = mkIdent (posOfNode ni) name (Name 0) - -Functions to generate AST objects from C template code - -> mkInputStream :: String -> InputStream -> mkInputStream s = inputStreamFromString s - -> constructDeclFromC :: NodeInfo -> String -> CDecl -> constructDeclFromC ni s = -> let result = (execParser_ extDeclP (mkInputStream s) $ posOfNode ni) in -> case result of -> -- parse succeeded, so we return the declaration -> (Right (CDeclExt decl)) -> decl -> -- parse failed, so we assert fail for now. The CDecl empty constructor is needed to match types -> (Left pe) -> trace ("Parse Error in:\n\n" ++ ("mkStmtFromC failed: " ++ s) ++ (show pe)) $ assert False (CDecl [] [] ni) - -> mkStmtFromC :: NodeInfo -> String -> CStat -> mkStmtFromC ni s = -> let result = (execParser_ statementP (mkInputStream s) $ posOfNode ni) in -> case result of -> -- parse succeeded, so we return the statement -> (Right stmt) -> stmt -> (Left pe) -> trace ("Parse Error in:\n\n" ++ -> ("mkStmtFromC failed: " ++ s) ++ -> (show pe)) $ assert False (CBreak ni) - -> mkStmtsFromCLines :: NodeInfo -> String -> [CStat] -> mkStmtsFromCLines ni lines = [mkStmtFromC ni lines] - -> data MacroParser = MacroParser { compiler :: String, macheader :: FilePath, typedefs :: [Ident] } - -> addTypeIdents :: [Ident] -> MacroParser -> MacroParser -> addTypeIdents idents m = m { typedefs = idents ++ (typedefs m) } - -> getTypeIdents :: String -> FilePath -> [String] -> IO [Ident] -> getTypeIdents compiler macheader gccopts = do -> arp_debug "getTypeIdents:" -> rfile <- myCPP compiler (Just macheader) gccopts (Left "\n") -> result <- readFile rfile -> -- removeFile rfile -> -- run the C parser on the result stream -> let parsed = execParser_ translUnitP (mkInputStream result) $ initPos macheader -> -- now get the typedefs as idents from the external declarations -> case parsed of -> (Right (CTranslUnit [] _)) -> return [] -> (Right (CTranslUnit decls _)) -> return $ getTypedefIdentsFromDecls decls -> (Left pe) -> trace ("Error parsing declarations from header " ++ macheader ++ ": " ++ (show pe)) (assert False (return [])) - -> mkParser :: String -> FilePath -> [String] -> IO MacroParser -> mkParser compiler macheader gccopts = do -> arp_debug ("mkParser: " ++ "macheader: " ++ macheader) -> types <- getTypeIdents compiler macheader gccopts -> arp_debug ("types: " ++ (show(types))) -> result <- myCPP compiler Nothing (["-imacros", macheader, "-dM", "-P"] ++ gccopts) (Left "\n") -> (errorFP, errorH) <- tmpHandle -> (outFile, outH) <- tmpHandle -> pid <- runProcess "grep" ["-v", "__STDC", result] Nothing Nothing Nothing (Just outH) (Just errorH) -> ec <- waitForProcess pid -> case ec of -> ExitSuccess -> do -> hClose errorH -> -- removeFile errorFP -> return outFile -> ExitFailure e -> do -> isOpen <- hIsOpen errorH -> errorH <- if not isOpen then openFile errorFP ReadMode else return errorH -> errs <- hGetContents errorH -> -- removeFile errorFP -> trace ("Preprocessor failed with return code: " ++ (show e) ++ "\n" ++ errs ++ "\n\n") $ assert False $ return "dummy_file.c" -> hClose errorH -> return "dummy_file.c" -> arp_debug ("mkParser constructs outFile: " ++ outFile) -> return $ MacroParser compiler outFile types - --- > getInfo :: NodeInfo -> String --- > getInfo info = ( - -RETURNS code as string - -> doCPPWithParser :: MacroParser -> NodeInfo -> String -> IO String -> doCPPWithParser p ni input = do -> arp_debug ("doCPPWithParser... ") -> -- arp_debug ( " on: " ++ (show (macheader p)) ++ " " ++ (fileAndLine ni)) -> fp <- myCPP (compiler p) (Just $ macheader p) ["-P"] (Left input) -> s <- readFile fp -> -- removeFile fp -> arp_debug ("doCPPWithParser: fp: " ++ fp) -> -- arp_debug ("OUTPUT:\n--\n" ++ s ++ "\n--\n") -> return s - -> doStmtPP :: MacroParser -> NodeInfo -> String -> [String] -> IO String -> doStmtPP p ni macro params = do -> arp_debug "doStmtPP:" -> let paramsStr [] = "" -> paramsStr pp = foldl1 ((++) . (++ ",")) pp -> macBody = "void tmp_triton_macro_fun(void) {\n" ++ macro ++ "(" ++ (paramsStr params) ++ ")\n}\n" - -> -- putStrLn $ join $ map (\(a,b) -> "-D" ++ a ++ "=" ++ b) defines -> doCPPWithParser p ni macBody - -> swapNodeInfo :: NodeInfo -> NodeInfo -> NodeInfo -> swapNodeInfo newni oldni = newni - -> swapNodeInfoInStmts :: NodeInfo -> [CStat] -> [CStat] -> swapNodeInfoInStmts ni stmts = map (everywhere (mkT $ swapNodeInfo ni)) stmts - -> mkStmtsFromCPPMacro :: MacroParser -> NodeInfo -> String -> [String] -> [CStat] -> mkStmtsFromCPPMacro p ni macro params = swapNodeInfoInStmts ni stmts -> where preproc = unsafePerformIO $ doStmtPP p ni macro params -> parsedResult = fmap fst $ (execParser translUnitP (mkInputStream preproc) (posOfNode ni) (typedefs p) newNameSupply) -> stmts = case parsedResult of -> -- parse succeeded, so we return the statement -> (Right (CTranslUnit [] _)) -> [] -> (Right (CTranslUnit edecls _)) -> case (last edecls) of -> (CFDefExt (CFunDef _ _ _ (CCompound _ bstmts _) _)) -> getStmtsFromBlockItems bstmts -> _ -> [] -> (Left pe) -> trace ("Error generating function definitions from macro '" ++ -> macro ++ "': " ++ preproc ++ (show pe)) -> (assert False []) - -> mkExprFromCPPMacro :: MacroParser -> NodeInfo -> String -> [String] -> Maybe CExpr -> mkExprFromCPPMacro p ni macro params = e -> where [s] = mkStmtsFromCPPMacro p ni macro params -> (CExpr e _) = s - -> declDummyName :: String -> declDummyName = "__dummy_decl" - -doDeclPP p: Parser - ni: Node Info - macro: String macro name - params: List of String: - AER_MK_STATIC_OP_DECL: (user function name (fname)) - AER_MK_SERVICE_DECL: (user function name (fname)) - (return type, user function name, RPC input, RPC output) - -> doDeclPP :: MacroParser -> NodeInfo -> String -> [String] -> IO String -> doDeclPP p ni macro params = do -> arp_debug ("doDeclPP: macro: " ++ macro ++ " " ++ (fileAndLine ni)) -> let paramsStr [] = "" -> paramsStr pp = foldl1 ((++) . (++ ",")) pp -> macBody = "int " ++ declDummyName ++ ";\n" ++ -> macro ++ "(" ++ (paramsStr params) ++ ")\n" -> arp_debug ("doDeclPP: paramsStr: " ++ (paramsStr params)) -> doCPPWithParser p ni macBody - -> getDeclsAfterDummy :: [CExtDecl] -> [CDecl] -> getDeclsAfterDummy edecls = map getd $ reverse $ takeWhile dname (reverse edecls) -> where getd (CDeclExt d) = d -> dname (CDeclExt d) = (/=) declDummyName $ identToString (getCDeclName d) -> dname _ = False - -> swapNodeInfoInDecls :: NodeInfo -> [CDecl] -> [CDecl] -> swapNodeInfoInDecls ni decls = map (everywhere (mkT $ swapNodeInfo ni)) decls - -mkDeclsFromCPPMacro p: Parser - ni: NodeInfo - macro: String name - params: String - -> mkDeclsFromCPPMacro :: MacroParser -> NodeInfo -> String -> [String] -> [CDecl] -> mkDeclsFromCPPMacro p ni macro params = -> swapNodeInfoInDecls ni decls -> where preproc = unsafePerformIO $ (arp_trace ("mkDeclsFromCPP: " ++ macro ++ " " ++ (fileAndLine ni))) doDeclPP p ni macro params -> parsedResult = fmap fst $ (execParser translUnitP (mkInputStream preproc) (posOfNode ni) (typedefs p) newNameSupply) -> decls = case parsedResult of -> -- parse succeeded, so we return the statement -> (Right (CTranslUnit [] _)) -> [] -> (Right (CTranslUnit edecls _)) -> getDeclsAfterDummy edecls -> (Left pe) -> trace ("Error generating function definitions from macro!\n" ++ -> "macro: '" ++ macro ++ "'\n" ++ -> "evaluates to: \n--\n" ++ preproc ++ "--\n" ++ -> "error from parser: \n" ++ (show pe)) -> (assert False []) - - AER_MK_SERVICE_FNDEF - __fname__, \ - __intype__, \ - __incanontype__, \ - __inname__, \ - __outtype__, \ - __outcanontype__, \ - __outname__, \ - __ptr__ \ - -> doFunDefPP :: MacroParser -> NodeInfo -> String -> [String] -> IO String -> doFunDefPP p ni macro params = do -> arp_debug ("doFunDefPP: " ++ macro) -> let paramsStr [] = "" -> paramsStr pp = foldl1 ((++) . (++ ",")) pp -> macBody = "int " ++ declDummyName ++ ";\n" ++ -> macro ++ "(" ++ (paramsStr params) ++ ")\n" -> arp_debug ("doFunDefPP: paramsStr: " ++ (paramsStr params)) -> doCPPWithParser p ni macBody - -> getFunDefDeclsAfterDummy :: [CExtDecl] -> [CExtDecl] -> getFunDefDeclsAfterDummy edecls = reverse $ takeWhile dname (reverse edecls) -> where dname (CDeclExt d) = (/=) declDummyName $ identToString (getCDeclName d) -> dname _ = True - -> swapStmts :: String -> [CStat] -> CStat -> CStat -> swapStmts stmtName insert a@(CLabel name _ _ ni) -> | (identToString name) == "__AESOP_BLOCK_INSERT_HERE__" = mkCompoundStmt (Just stmtName) insert ni -> | otherwise = a -> swapStmts _ _ a = a - -> insertStmtBlock :: (Data a) => String -> [CStat] -> a -> a -> insertStmtBlock name insert a = everywhere (mkT $ swapStmts name insert) a - -> swapNodeInfoInExtDecls :: NodeInfo -> [CExtDecl] -> [CExtDecl] -> swapNodeInfoInExtDecls ni edecls = map (everywhere (mkT $ swapNodeInfo ni)) edecls - -mkFunDefFromCPPMacro params - -> mkFunDefFromCPPMacro :: MacroParser -> NodeInfo -> String -> [String] -> String -> [CStat] -> [CExtDecl] -> mkFunDefFromCPPMacro p ni macro params insertName insertStmts = (arp_trace ("mkFunDefFromCPPMacro: " ++ (fileAndLine ni) ++ " " ++ (show params))) $ swapNodeInfoInExtDecls ni [completedFunDef] -> where preproc = unsafePerformIO $ doFunDefPP p ni macro params -> parsedResult = fmap fst $ (execParser translUnitP (mkInputStream preproc) (posOfNode ni) (typedefs p) newNameSupply) -> decls = case parsedResult of -> -- parse succeeded, so we return the statement -> (Right (CTranslUnit [] _)) -> [] -> (Right (CTranslUnit edecls _)) -> getFunDefDeclsAfterDummy edecls -> (Left pe) -> trace ("Error generating function definitions from macro '" ++ macro ++ "': " ++ preproc ++ (show pe)) -> (assert False []) - -> [fdef] = assert ((length decls) == 1) $ decls -> completedFunDef = insertStmtBlock insertName insertStmts fdef - -> mkCompoundStmt :: Maybe String -> [CStat] -> NodeInfo -> CStat -> mkCompoundStmt (Just label) stmts ni = CLabel (newIdent label ni) (CCompound [] (map CBlockStmt stmts) ni) [] ni -> mkCompoundStmt Nothing stmts ni = CCompound [] (map CBlockStmt stmts) ni - -> mkCompoundWithDecls :: Maybe String -> [CDecl] -> [CStat] -> NodeInfo -> CStat -> mkCompoundWithDecls (Just label) decls stmts ni = -> CCompound [newIdent label ni] ((map CBlockDecl decls) ++ (map CBlockStmt stmts)) ni -> mkCompoundWithDecls Nothing decls stmts ni = -> CCompound [] ((map CBlockDecl decls) ++ (map CBlockStmt stmts)) ni - -> constructExprFromC :: NodeInfo -> String -> CExpr -> constructExprFromC ni s = -> let result = (execParser_ expressionP (mkInputStream s) $ posOfNode ni) in -> case result of -> -- parse succeeded, so we return the expression -> (Right expr) -> expr -> (Left pe) -> trace ("Error constructing expression:\n" ++ s ++ (show pe)) (assert False (CVar (newIdent "blah" ni) ni)) - -funDefDecl creates the return type declaration for a given function. The parameters to funDefDecl are -the function definition and the name of the return type. For example, a function defined as: - -int ** myFunc(int a); - -would generate a CDecl like this: - -int ** ret; - -> mkRetDecl :: (CTypeSpec, [CDerivedDeclr]) -> String -> NodeInfo -> [CDecl] -> mkRetDecl ((CVoidType _), []) name ni = [] -> mkRetDecl rtype name ni = [CDecl [(CTypeSpec (fst rtype))] -> [(Just (CDeclr (Just $ newIdent name ni) (snd rtype) Nothing [] ni), -> Nothing, -> Nothing)] -> ni] - -> mkRetDeclFromFunDef :: CFunDef -> String -> [CDecl] -> mkRetDeclFromFunDef f name = -> mkRetDecl rtype name (nodeInfo f) -> where rtype = fromJust $ getFunDefReturnMaybe f - -Functions for building struct members from scratch - -Make a declarator: <derived> name; -Examples: - -**foo; - -mkCDeclr "foo" [CPtrDeclr, CPtrDeclr] ... - -(*myFunPtr)(int bar, char *baz); - -mkCDeclr "myFunPtr" [CPtrDeclr, (CFunDeclr (Right [(CDecl [CTypeSpec CIntType] [("bar" ...)]), ...]) ...)] - -> mkCDeclr :: String -> [CDerivedDeclr] -> CDeclr -> mkCDeclr name derived = -> let ni = nodeInfo $ head derived -> in CDeclr (Just (newIdent name ni)) derived Nothing [] ni - -> mkIdentCDeclr :: Ident -> [CDerivedDeclr] -> CDeclr -> mkIdentCDeclr name derived = -> let ni = nodeInfo $ head derived -> in CDeclr (Just name) derived Nothing [] ni - -> mkAnonCDeclr :: [CDerivedDeclr] -> CDeclr -> mkAnonCDeclr derived = -> let ni = nodeInfo $ head derived -> in CDeclr Nothing derived Nothing [] ni - -Make a declaration: type <derived> name; -Examples: - -int **foo; - -mkCDecl CIntType [CPtrDeclr, CPtrDeclr] "foo" - -struct myStruct bar; - -mkCDecl (CSUType (CStruct CStructTag "myStruct" Nothing [] ...) ...) [] "bar" - -myType *baz; - -mkCDecl (CTypeDef "myType") [CPtrDeclr] "baz" - -> mkCDecl :: CTypeSpec -> [CDerivedDeclr] -> String -> NodeInfo -> CDecl -> mkCDecl mtype derives name ni = CDecl [CTypeSpec mtype] [(Just $ mkCDeclr name derives, Nothing, Nothing)] ni - -> mkIdentCDecl :: CTypeSpec -> [CDerivedDeclr] -> Ident -> NodeInfo -> CDecl -> mkIdentCDecl mtype derives name ni = CDecl [CTypeSpec mtype] [(Just $ mkIdentCDeclr name derives, Nothing, Nothing)] ni - -> mkAnonCDecl :: CTypeSpec -> [CDerivedDeclr] -> NodeInfo -> CDecl -> mkAnonCDecl mtype derives ni = CDecl [CTypeSpec mtype] [(Just $ mkAnonCDeclr derives, Nothing, Nothing)] ni - -> mkAnonFromDecl :: CDecl -> CDecl -> mkAnonFromDecl (CDecl specs inits ni) = (CDecl specs (map stripName inits) ni) -> where stripName ((Just (CDeclr (Just n) deriveds strlit attrs sni)), init, ex) = -> ((Just (CDeclr Nothing deriveds strlit attrs sni)), init, ex) - -> mkConstCDecl :: CTypeSpec -> [CDerivedDeclr] -> String -> NodeInfo -> CDecl -> mkConstCDecl mtype derives name ni = CDecl [CTypeQual (CConstQual ni), CTypeSpec mtype] [(Just $ mkCDeclr name derives, Nothing, Nothing)] ni - -> genCDecl :: String -> String -> NodeInfo -> CDecl -> genCDecl typedef name ni = mkCDecl (CTypeDef (newIdent typedef ni) ni) [] name ni - -> addBlockingTQ :: CDecl -> CDecl -> addBlockingTQ (CDecl specs inits ni) = CDecl (CTypeQual ((CBlocking ni)):specs) inits ni - -> mkFunPtrsStruct :: String -> String -> [(String, String)] -> Bool -> NodeInfo -> CExtDecl -> mkFunPtrsStruct stype sname fields static ni = CDeclExt decl -> where decl = CDecl (staticSpec ++ [tspec]) declrs ni -> tspec = CTypeSpec (CSUType (CStruct CStructTag (Just $ newIdent stype ni) Nothing [] ni) ni) -> staticSpec = if static then [CStorageSpec $ CStatic ni] else [] -> declrs = [(Just vname, Just inits, Nothing)] -> vname = CDeclr (Just (newIdent sname ni)) [] Nothing [] ni -> inits = CInitList [ ( [CMemberDesig (newIdent field ni) ni], CInitExpr (CVar (newIdent fun ni) ni) ni ) | (field, fun) <- fields ] ni - -Functions for building a struct from the name and array of members - -param1Decl can be created with mkCDecl. - -> mkStructDef :: Ident -> [CDecl] -> NodeInfo -> CTypeSpec -> mkStructDef name members ni = CSUType (CStruct CStructTag (Just name) (Just members) [] ni) ni - -> mkStructDefDecl :: String -> [CDecl] -> NodeInfo -> CDecl -> mkStructDefDecl name members ni = -> CDecl [CTypeSpec $ mkStructDef (newIdent name ni) members ni] [(Nothing, Nothing, Nothing)] ni - -> mkStructDecl :: String -> String -> NodeInfo -> CDecl -> mkStructDecl tname name ni = mkCDecl (CSUType (CStruct CStructTag (Just $ newIdent tname ni) -> Nothing -> [] -> ni) ni) [] name ni - -> mkStructPtrDecl :: String -> String -> NodeInfo -> CDecl -> mkStructPtrDecl tname name ni = mkCDecl (CSUType (CStruct CStructTag (Just $ newIdent tname ni) -> Nothing -> [] -> ni) ni) [(CPtrDeclr [] ni)] name ni - -> addPtrToDeclarators :: (Maybe CDeclr, Maybe CInit, Maybe CExpr) -> (Maybe CDeclr, Maybe CInit, Maybe CExpr) -> addPtrToDeclarators (Just (CDeclr name d asm attrs ni), i, e) = -> (Just (CDeclr name (d ++ [CPtrDeclr [] ni]) asm attrs ni), i, e) -> addPtrToDeclarators a = a - -> addPtrDeclr :: CDecl -> CDecl -> addPtrDeclr (CDecl specs declarators ni) = (CDecl specs ((addPtrToDeclarators (head declarators)):(tail declarators)) ni) - -> mkAnonStructPtrDecl :: String -> NodeInfo -> CDecl -> mkAnonStructPtrDecl tname ni = mkAnonCDecl (CSUType (CStruct CStructTag (Just $ newIdent tname ni) Nothing [] ni) ni) -> [(CPtrDeclr [] ni)] ni -> mkIntDecl :: String -> NodeInfo -> CDecl -> mkIntDecl name ni = mkCDecl (CIntType ni) [] name ni - -From a name list of parameters, and return type, construct an external declaration struct - -Example: - -struct myStruct -{ - param1type param1; - param2type param2; -}; - -genStructExtDecl "myStruct" [param1Decl, param2Decl] - -> genStructExtDecl :: String -> [CDecl] -> NodeInfo -> CExtDecl -> genStructExtDecl name members ni = CDeclExt $ mkStructDefDecl name members ni - -Make a function pointer declarator: (*) (params); - -> mkFunPtrDeclr :: [CDecl] -> [CDerivedDeclr] -> mkFunPtrDeclr params = -> let ni = nodeInfo $ head params -> in [CPtrDeclr [] ni, CFunDeclr (Right (params, False)) [] ni] - -Make the function pointer: void (* name) (params); - -> mkFunPtrDecl :: String -> CTypeSpec -> [CDecl] -> CDecl -> mkFunPtrDecl name retType params = -> let ni = nodeInfo retType -> in mkCDecl retType (mkFunPtrDeclr params) name ni - -> mkFunPtrDerivedDecl :: String -> CTypeSpec -> [CDerivedDeclr] -> [CDecl] -> CDecl -> mkFunPtrDerivedDecl name retType retDerived params = -> let ni = nodeInfo retType -> in mkCDecl retType ((mkFunPtrDeclr params) ++ retDerived) name ni - -> mkVoidFunPtr :: String -> [CDecl] -> CDecl -> mkVoidFunPtr name params = -> let ni = nodeInfo $ head params -> in mkFunPtrDecl name (CVoidType ni) params - -> mkFunDeclr :: [CDecl] -> NodeInfo -> CDerivedDeclr -> mkFunDeclr params ni = -> CFunDeclr (Right (params, False)) [] ni - -> mkFunDecl :: String -> CTypeSpec -> [CDerivedDeclr] -> [CDecl] -> CDecl -> mkFunDecl fname returnType returnDerived params = -> let ni = nodeInfo returnType -> in mkCDecl returnType ([mkFunDeclr params ni] ++ returnDerived) fname ni - -> mkIdentFunDecl :: Ident -> CTypeSpec -> [CDerivedDeclr] -> [CDecl] -> CDecl -> mkIdentFunDecl fname returnType returnDerived params = -> let ni = nodeInfo returnType -> in mkIdentCDecl returnType ([mkFunDeclr params ni] ++ returnDerived) fname ni - -> mkFunDeclWithDeclSpecs :: String -> CTypeSpec -> [CDerivedDeclr] -> [CDeclSpec] -> [CDecl] -> CDecl -> mkFunDeclWithDeclSpecs fname rType rDerived declSpecs params = -> let ni = nodeInfo rType -> in CDecl (declSpecs ++ [CTypeSpec rType]) -> [(Just $ mkCDeclr fname ([mkFunDeclr params ni] ++ rDerived), -> Nothing, Nothing)] ni - -> mkFunPtrDeclWithDeclSpecs :: String -> CTypeSpec -> [CDerivedDeclr] -> [CDeclSpec] -> [CDecl] -> Maybe String -> CDecl -> mkFunPtrDeclWithDeclSpecs fname rType rDerived declSpecs params initName = -> let ni = nodeInfo rType -> in CDecl (declSpecs ++ [CTypeSpec rType]) -> [(Just $ mkCDeclr fname ([CPtrDeclr [] ni, mkFunDeclr params ni] ++ rDerived), -> (if isJust initName -> then Just (CInitExpr (CVar (newIdent (fromJust initName) ni) ni) ni) -> else Nothing), Nothing)] ni - -Remove the initializer from the declarator. For example: - -int a = 10; - -becomes: - -int a; - -We don't remove the initialization list, so: - -int a[2] = {1, 2}; - -remains. - -> removeInitFromDeclr :: (Maybe CDeclr, Maybe CInit, Maybe CExpr) -> (Maybe CDeclr, Maybe CInit, Maybe CExpr) -> removeInitFromDeclr ((,,) declr (Just (CInitExpr expr _)) cexpr) = (declr, Nothing, cexpr) -> removeInitFromDeclr t = t - -> removeInitFromDecl :: CDecl -> CDecl -> removeInitFromDecl (CDecl declspecs declrs ni) = (CDecl declspecs (map removeInitFromDeclr declrs) ni) - -> getInitFromDeclr :: (Maybe CDeclr, Maybe CInit, Maybe CExpr) -> Maybe CExpr -> getInitFromDeclr ((,,) _ (Just (CInitExpr initexpr _)) _) = Just initexpr -> getInitFromDeclr _ = Nothing - -> getNameFromDeclr :: (Maybe CDeclr, Maybe CInit, Maybe CExpr) -> Maybe Ident -> getNameFromDeclr ((,,) (Just (CDeclr name _ _ _ _)) _ _) = name -> getNameFromDeclr _ = Nothing - -Get the init statements from a declaration. As an example, the declaration: - -int a, b = 3, c = 4; - -results in the statements: - -b = 3; -c = 4; - -> getInitStmtsFromDecl :: CDecl -> [CStat] -> getInitStmtsFromDecl d@(CDecl declspecs declrs ni) = do -> declr <- declrs -> let name = getNameFromDeclr declr -> init = getInitFromDeclr declr -> ni = nodeInfo d -> if isJust init && isJust name then (return $ mkAssignStmt (CVar (fromJust name) ni) (fromJust init)) else [] - -> getInitsFromDecls :: [CDecl] -> [CStat] -> getInitsFromDecls ds = join $ map getInitStmtsFromDecl ds - -> removeVarNameFromDeclr :: CDeclr -> CDeclr -> removeVarNameFromDeclr (CDeclr name indirections asm attrs ni) = -> (CDeclr Nothing indirections asm attrs ni) - -> removeVarFromDecl :: CDecl -> CDecl -> removeVarFromDecl (CDecl specs indirs ni) = (CDecl specs (map stripVar indirs) ni) where -> stripVar i@(cd, ci, ce) = -> if isJust cd -> then (Just (removeVarNameFromDeclr $ fromJust $ cd), Nothing, Nothing) -> else i - -Given a lhs expression and rhs expression, construct the statement: lhs = rhs; - -> mkAssignStmt :: CExpr -> CExpr -> CStat -> mkAssignStmt lexpr rexpr = -> let ni = nodeInfo lexpr -> in (CExpr (Just (CAssign CAssignOp lexpr rexpr ni)) ni) - -> mkFunCall :: CExpr -> CExpr -> [CExpr] -> CStat -> mkFunCall ret call params = -> let ni = nodeInfo ret -> in (CExpr (Just (CAssign CAssignOp ret (CCall call params ni) ni)) ni) - -> addParamsToFunCall :: CStat -> [CExpr] -> CStat -> addParamsToFunCall -> (CExpr (Just (CAssign -> CAssignOp -> ret -> (CCall call params ni1) ni2)) ni3) addParams = -> (CExpr (Just (CAssign -> CAssignOp -> ret -> (CCall call (params ++ addParams) ni1) ni2)) ni3) -> addParamsToFunCall -> (CExpr (Just (CCall call params ni1)) ni2) addParams = -> (CExpr (Just (CCall call (params ++ addParams) ni1)) ni2) - -> mkVar :: String -> NodeInfo -> CExpr -> mkVar s ni = (CVar (newIdent s ni) ni) - -Tack on a struct pointer variable to an expression (assuming its a CVar). For example, with an expression of "blah" - -addStructPtr "params" expr - -will make: - -params->blah - -> addStructPtr :: String -> CExpr -> CExpr -> addStructPtr sname (CVar name ni) = CMember (CVar (newIdent sname ni) ni) name True ni -> addStructPtr _ expr = expr - -> addAddrOp :: CExpr -> CExpr -> addAddrOp c = (CUnary CAdrOp c (nodeInfo c)) - -addStructPtrPrefix "ctl" "params" (CVar "blah") - -will make: - -ctl->params->blah - -> addStructPtrPrefix :: String -> String -> CExpr -> CExpr -> addStructPtrPrefix sname pname (CVar name ni) = CMember (CMember (CVar (newIdent sname ni) ni) (newIdent pname ni) True ni) name True ni -> addStructPtrPrefix _ _ e = e - -Same as above, but without the second pointer deref: - -ctl->params.blah - -> addStructPrefix :: String -> String -> CExpr -> CExpr -> addStructPrefix sname pname (CVar name ni) = -> CMember (CMember (CVar (newIdent sname ni) ni) -> (newIdent pname ni) True ni) name False ni - -ctl->params1.params2.blah - -> addStructPrefixPrefix :: String -> String -> String -> CExpr -> CExpr -> addStructPrefixPrefix sname p1name p2name (CVar name ni) = -> CMember (CMember (CMember (CVar (newIdent sname ni) ni) -> (newIdent p1name ni) True ni) -> (newIdent p2name ni) False ni) name False ni - -ctl->params1->params2.blah - -> addStructPtrPrefixPrefix :: String -> String -> String -> CExpr -> CExpr -> addStructPtrPrefixPrefix sname p1name p2name (CVar name ni) = -> CMember (CMember (CMember (CVar (newIdent sname ni) ni) (newIdent p1name ni) True ni) (newIdent p2name ni) True ni) name False ni - -Take an assign statement, and insert a struct pointer prefix to the lhs. For example: - -addStructToAssign "params" "mymem" "funcall(a, 123)" - -results in the statement: params.mymem = funcall(a, 123); - -> addStructToAssign :: String -> String -> Ident -> CExpr -> CStat -> addStructToAssign sname pname lvar rexpr = -> let ni = nodeInfo rexpr -> in (CExpr (Just (CAssign CAssignOp (addStructPrefix sname pname (CVar lvar ni)) rexpr ni)) ni) - -Take an assign statement, and insert a struct pointer prefix to the lhs. For example: - -addStructPtrToAssign "params" "mymem" "funcall(a, 123)" - -results in the statement: params->mymem = funcall(a, 123); - -> addStructPtrToAssign :: String -> String -> Ident -> CExpr -> CStat -> addStructPtrToAssign sname pname lvar rexpr = -> let ni = nodeInfo rexpr -> in (CExpr (Just (CAssign CAssignOp (addStructPtrPrefix sname pname (CVar lvar ni)) rexpr ni)) ni) - -> mkFunDef :: (CTypeSpec, [CDerivedDeclr]) -> [CDeclSpec] -> String -> [CDecl] -> CStat -> CExtDecl -> mkFunDef (rtype, derivedPtrs) dspecs fname params stmts = -> let ni = if null params then nodeInfo stmts else nodeInfo $ head params -> in CFDefExt (CFunDef (dspecs ++ [(CTypeSpec rtype)]) -> (CDeclr (Just $ newIdent fname ni) -> (derivedPtrs ++ -> [(CFunDeclr (Right (params, False)) [] ni)]) -> Nothing [] ni) -> [] stmts ni) - -> addDeclSpecs :: CExtDecl -> [CDeclSpec] -> CExtDecl -> addDeclSpecs (CFDefExt (CFunDef s d dls stmt ni)) newspecs = CFDefExt $ CFunDef (newspecs ++ s) d dls stmt ni - -> mkStaticFunDecl :: CTypeSpec -> String -> [CDecl] -> CExtDecl -> mkStaticFunDecl return fname params = -> let ni = nodeInfo return -> in CDeclExt (CDecl [(CStorageSpec (CStatic ni)), -> (CTypeSpec return)] -> [((Just $ mkCDeclr fname [mkFunDeclr params ni]), Nothing, Nothing)] ni) - -> mkStaticFunDef :: CTypeSpec -> String -> [CDecl] -> CStat -> CExtDecl -> mkStaticFunDef return fname params stmts = -> let ni = if null params then nodeInfo stmts else nodeInfo $ head params -> in CFDefExt (CFunDef [(CStorageSpec (CStatic ni)), (CTypeSpec return)] -> (CDeclr (Just $ newIdent fname ni) -> [(CFunDeclr (Right (params, False)) [] ni)] -> Nothing [] ni) -> [] stmts ni) - -> mkIfCondition :: Maybe CExpr -> CExpr -> mkIfCondition mexpr = if isJust mexpr then fromJust mexpr else CConst (CIntConst (cInteger 1) $ nodeInfo $ fromJust mexpr) - -> mkIfStmt :: Maybe CExpr -> [CStat] -> CStat -> mkIfStmt cond stmts = CIf (mkIfCondition cond) (CCompound [] (map CBlockStmt stmts) $ nodeInfo $ fromJust cond) Nothing $ nodeInfo $ fromJust cond - -> mkIfElseStmt :: Maybe CExpr -> [CStat] -> [CStat] -> CStat -> mkIfElseStmt cond ifstmts elsestmts = -> CIf (mkIfCondition cond) -> (CCompound [] (map CBlockStmt ifstmts) $ nodeInfo $ fromJust cond) -> (Just (CCompound [] (map CBlockStmt elsestmts) $ nodeInfo $ fromJust cond)) $ nodeInfo $ fromJust cond - -> mkIntConst :: Integer -> NodeInfo -> CExpr -> mkIntConst v ni = CConst (CIntConst (cInteger v) ni) - -> mkGoto :: String -> NodeInfo -> CStat -> mkGoto s ni = (CGoto (newIdent s ni) ni) - -> mkLabel :: String -> [CStat] -> NodeInfo -> CStat -> mkLabel s stmts ni = (CLabel (newIdent s ni) (CCompound [] (map CBlockStmt stmts) ni) [] ni) - -> canonCDecl :: CDecl -> String -> canonCDecl (CDecl declspecs derived _) = -> let t = getTypeSpec declspecs -> tstr = case t of -> (CSUType (CStruct CStructTag (Just n) _ _ _) _) -> "struct_" ++ (identToString n) -> (CTypeDef n _) -> identToString n -> (CSUType (CStruct CUnionTag (Just n) _ _ _) _) -> "union_" ++ (identToString n) -> (CEnumType (CEnum (Just n) _ _ _) _) -> "enum_" ++ (identToString n) -> _ -> show $ pretty t -> ptrs = filter isDerivedPtr (join $ map getDerivedDeclrs derived) -> ptrStr = concat [ "_ptr" | p <- ptrs ] -> in tstr ++ ptrStr - -> removeAPtr :: CDecl -> CDecl -> removeAPtr (CDecl sp derived n) = (CDecl sp nd n) -> where nd = map remP derived -> remP (declr, init, expr) = (rem declr, init, expr) -> rem Nothing = Nothing -> rem (Just (CDeclr name deriveds lit attrs ni)) = (Just (CDeclr name (remd deriveds) lit attrs ni)) -> remd (d:ds) = if isDerivedPtr d then ds else (d:(remd ds)) -> remd [] = [] - -> mkCastStmt :: String -> CDecl -> String -> NodeInfo -> CStat -> mkCastStmt l t r ni = CExpr (Just (CAssign CAssignOp (CVar (newIdent l ni) ni) (CCast t (CVar (newIdent r ni) ni) ni) ni)) ni - diff --git a/code/src/aesop/parser/CParse.lhs b/code/src/aesop/parser/CParse.lhs deleted file mode 100644 index 4390e5f..0000000 --- a/code/src/aesop/parser/CParse.lhs +++ /dev/null @@ -1,470 +0,0 @@ -> module CParse where -> import Language.C -> import Language.C.Data.Node -> import Language.C.Data.Position -> import Language.C.Data.Ident -> import Language.C.System.GCC -- preprocessor used -> import System.Environment -> import Data.Typeable -> import Data.Maybe -> import Data.Either -> import Data.HashTable -> import Data.Generics -> import Data.Generics.Schemes -> import Control.Monad.State -> import qualified Data.ByteString -> import qualified Data.ByteString.Char8 -> import Control.Exception -> import Data.List -> import System.IO.Unsafe -> import Debug.Trace - -Add a first function for triples - -> tfst :: (a,b,c) -> a -> tfst ((,,) a _ _) = a - -Get the line number and column string from a node info. For example, a variable at line 123, column 34 will result -in the string: - -_123_34_ - -> getPosVarStr :: NodeInfo -> String -> getPosVarStr (NodeInfo p _ _) = "_" ++ (show $ posRow p) ++ "_" ++ (show $ posColumn p) ++ "_" - -This is useful for removing the names (and initializers) from a declaration, making it -a declaration without a name (good for return types, etc.). - -> emptyDeclrList :: CDecl -> CDecl -> emptyDeclrList (CDecl specifiers _ ni) = CDecl specifiers [] ni - -Get the declarators from a list of triples - -> getCDeclrsFromList :: [(Maybe CDeclr, Maybe CInit, Maybe CExpr)] -> [CDeclr] -> getCDeclrsFromList cdeclrs = -> concatMap (\cd -> if isJust $ tfst cd then [fromJust $ tfst cd] else []) cdeclrs - - Get the name from a declarator - -> getCDeclrName :: CDeclr -> Ident -> getCDeclrName (CDeclr name _ _ _ _) = fromJust $ name - -Get the names from a declaration. For example: - -int a, b, c = 3; - -Will result in ["a", "b", "c"] - -> getCDeclNames :: CDecl -> [Ident] -> getCDeclNames (CDecl _ declrs _) = map getCDeclrName $ getCDeclrsFromList declrs - -> getCDeclName :: CDecl -> Ident -> getCDeclName (CDecl _ declrs _) = getCDeclrName $ d -> where (d:dls) = getCDeclrsFromList declrs - -> hasDeclrName :: CDecl -> Bool -> hasDeclrName (CDecl _ declrs _) = any (\(CDeclr name _ _ _ _) -> isJust name) $ getCDeclrsFromList declrs - -> splitDecls :: CDecl -> [CDecl] -> splitDecls (CDecl specifiers initdecls ni) = map (\dlr -> (CDecl specifiers [dlr] ni)) initdecls - -> isStructTypeSpec :: CDeclSpec -> Bool -> isStructTypeSpec (CTypeSpec (CSUType (CStruct _ _ _ _ _) _)) = True -> isStructTypeSpec _ = False - -> isTypeDefSpec :: CDeclSpec -> Bool -> isTypeDefSpec (CStorageSpec (CTypedef _)) = True -> isTypeDefSpec _ = False - -> isTypeDefTypeSpec :: CDeclSpec -> Bool -> isTypeDefTypeSpec (CTypeSpec (CTypeDef _ _)) = True -> isTypeDefTypeSpec _ = False - -> isVoidTypeSpec :: CDeclSpec -> Bool -> isVoidTypeSpec (CTypeSpec (CVoidType _)) = True -> isVoidTypeSpec _ = False - -> isStructDecl :: CDecl -> Bool -> isStructDecl (CDecl specs initdecls ni) = any isStructTypeSpec specs - -> isTypeDefDecl :: CDecl -> Bool -> isTypeDefDecl (CDecl (s:specs) initdecls ni) = isTypeDefSpec s -> isTypeDefDecl _ = False - -> getTypeDefIdents :: CTranslUnit -> [Ident] -> getTypeDefIdents (CTranslUnit extDecls _) = getTypedefIdentsFromDecls extDecls - -> getTypedefIdentsFromDecls :: [CExtDecl] -> [Ident] -> getTypedefIdentsFromDecls ds = concatMap getIdent $ concatMap getd ds -> where getd (CDeclExt d) = [d] -> getd _ = [] -> getIdent d@(CDecl specs inits ni) -> | isTypeDefDecl d = -> let (i:ids) = inits -> ((Just (CDeclr (Just id@(Ident tname _ _)) _ _ _ _)), _, _) = i -> in [id] -> | otherwise = [] - -> structSpecHasName :: CDeclSpec -> Bool -> structSpecHasName (CTypeSpec (CSUType (CStruct _ (Just _) _ _ _) _)) = True -> structSpecHasName _ = False - -> structSpecHasFields :: CDeclSpec -> Bool -> structSpecHasFields (CTypeSpec (CSUType (CStruct _ _ (Just fields) _ _) _)) = True -> structSpecHasFields _ = False - -> isTypeDefOfPredefStruct :: CDecl -> Bool -> isTypeDefOfPredefStruct (CDecl (s1:s2:[]) initdecls ni) = -> isTypeDefSpec s1 && isStructTypeSpec s2 && (not $ structSpecHasFields s2) -> isTypeDefOfPredefStruct _ = False - -> isTypeDefOfInlineStruct :: CDecl -> Bool -> isTypeDefOfInlineStruct (CDecl (s1:s2:[]) initdecls ni) = -> isTypeDefSpec s1 && isStructTypeSpec s2 && structSpecHasFields s2 -> isTypeDefOfInlineStruct _ = False - -> isTypeDefOfTypeDef :: CDecl -> Bool -> isTypeDefOfTypeDef (CDecl (s1:s2:[]) _ _) = -> isTypeDefSpec s1 && isTypeDefTypeSpec s2 -> isTypeDefOfTypeDef _ = False - -> getStructName :: CDeclSpec -> Maybe Ident -> getStructName (CTypeSpec (CSUType (CStruct _ n _ _ _) _)) = if isJust n then Just $ fromJust n else Nothing -> getStructName _ = Nothing - -> getStructTypeDefInfo :: CDecl -> Maybe (Maybe Ident, Ident) -> getStructTypeDefInfo d@(CDecl (s1:s2:[]) initdecls ni) -> | isTypeDefSpec s1 && isStructTypeSpec s2 = -> let (i:ids) = initdecls -> ((Just (CDeclr (Just ti@(Ident tname _ _)) _ _ _ _)), _, _) = i -> in Just (getStructName s2, ti) -> | otherwise = Nothing - -> getTypeDefTypeDefInfo :: CDecl -> Maybe (Ident, Ident) -> getTypeDefTypeDefInfo d@(CDecl (s1:s2:[]) initdecls ni) -> | isTypeDefSpec s1 && isTypeDefTypeSpec s2 = -> let (CTypeSpec (CTypeDef pi@(Ident pname _ _) _)) = s2 -> (i:ids) = initdecls -> ((Just (CDeclr (Just ti@(Ident tname _ _)) _ _ _ _)), _, _) = i -> in Just (pi, ti) -> | otherwise = Nothing - -> getStructInfo :: CDecl -> Maybe (Maybe Ident, [CDecl]) -> getStructInfo (CDecl s i n) = -> let (t:ts) = filter isStructTypeSpec s -> in case t of -> (CTypeSpec (CSUType (CStruct _ (Just n) (Just f) _ _) _)) -> Just (Just $ n, f) -> (CTypeSpec (CSUType (CStruct _ Nothing (Just f) _ _) _)) -> Just (Nothing, f) -> _ -> Nothing - -> getTypeName :: CDecl -> Maybe Ident -> getTypeName d@(CDecl s _ _) = -> let (sp:sps) = s -> in case sp of -> (CTypeSpec (CSUType (CStruct _ (Just n) _ _ _) _)) -> Just $ n -> _ -> Nothing - -> isVarDecl :: CDecl -> Bool -> isVarDecl d@(CDecl specs initdecls ni) = -> (not $ isFunDecl d) && (not $ any isTypeDefSpec specs) && hasDeclrName d - -> getVarName :: CDecl -> String -> getVarName d@(CDecl _ initdecls _) = vname -> where (i:ids) = initdecls -> ((Just (CDeclr (Just (Ident vname _ _)) _ _ _ _)), _, _) = i - -For a compound block item (part of a compound block of statements), is the item a declaration? - -> isFuncDecl :: CBlockItem -> Bool -> isFuncDecl (CBlockDecl decl) = True -> isFuncDecl _ = False - -For a compound block item (part of a compound block of statements), is the item a declaration? - -> getFuncDecl :: CBlockItem -> CDecl -> getFuncDecl (CBlockDecl decl) = decl - -For the array of compound block items, get the elements that are declarations - -> findFuncDecls :: [CBlockItem] -> [CDecl] -> findFuncDecls decls = [ getFuncDecl d | d <- decls, isFuncDecl d ] - -For the function derived declarator, get the function parameters - -> getCFunDeclrParams :: CDerivedDeclr -> [CDecl] -> getCFunDeclrParams (CFunDeclr (Right (declarations, isVariadic)) _ ni) = declarations - -> isTypeSpec :: CDeclSpec -> Bool -> isTypeSpec (CTypeSpec c) = True -> isTypeSpec _ = False - -> getTypeSpecFromDeclSpec :: CDeclSpec -> CTypeSpec -> getTypeSpecFromDeclSpec (CTypeSpec c) = c - -> getTypeSpec :: [CDeclSpec] -> CTypeSpec -> getTypeSpec declspecs = getTypeSpecFromDeclSpec $ fromJust $ find isTypeSpec declspecs - -> getTypeSpecFromDecl :: CDecl -> CTypeSpec -> getTypeSpecFromDecl (CDecl dspecs _ _) = getTypeSpec dspecs - -> isDerivedPtr :: CDerivedDeclr -> Bool -> isDerivedPtr (CPtrDeclr _ _) = True -> isDerivedPtr _ = False - -> getDerivedDeclrs :: (Maybe CDeclr, Maybe CInit, Maybe CExpr) -> [CDerivedDeclr] -> getDerivedDeclrs (Just (CDeclr _ derived _ _ _), _, _) = derived -> getDerivedDeclrs _ = [] - -> getDerivedPtrs :: CDeclr -> [CDerivedDeclr] -> getDerivedPtrs (CDeclr _ derivedDeclrs _ _ _) = -> if any isDerivedFun derivedDeclrs then -> tail $ snd $ break isDerivedFun derivedDeclrs -> else filter (not . isDerivedPtr) derivedDeclrs - -> getReturn :: CDecl -> (CTypeSpec, [CDerivedDeclr]) -> getReturn d@(CDecl specs declarators ni) = -> let declrs = getCDeclrsFromList declarators -> (d:ds) = declrs -> derivedPtrs = if null declrs then [] else getDerivedPtrs $ d -> in (getTypeSpec specs, derivedPtrs) - -> isDerivedFun :: CDerivedDeclr -> Bool -> isDerivedFun (CFunDeclr (Right ((,) params variadic)) _ _) -> | otherwise = True -> isDerivedFun _ = False - -> getDerivedFun :: CDeclr -> CDerivedDeclr -> getDerivedFun (CDeclr _ derivedDeclrs _ _ _) = fromJust $ find isDerivedFun derivedDeclrs - -> getDerivedFunParams :: CDerivedDeclr -> [CDecl] -> getDerivedFunParams (CFunDeclr (Right ((,) params variadic)) _ _) = params - -> getParams :: CDecl -> [CDecl] -> getParams (CDecl specs declarators ni) = -> let declrs = getCDeclrsFromList declarators -> (d:ds) = declrs -> in if null declrs then [] else getDerivedFunParams $ getDerivedFun $ d - -split a function declaration into tuple: (function-name, (return-type, return-derived-list), function-parameters) - -> splitFunDecl :: CDecl -> (Ident, (CTypeSpec, [CDerivedDeclr]), [CDecl]) -> splitFunDecl decl = (dn, getReturn decl, getParams decl) -> where (dn:dns) = getCDeclNames decl - -> getFunInfo :: CFunDef -> (Ident, (CTypeSpec, [CDerivedDeclr]), [CDecl]) -> getFunInfo f = (getFunDefIdent f, fromJust (getFunDefReturnMaybe f), getFunDefParams f) - -Get the name of a function definition - -> getFunDefName :: CFunDef -> String -> getFunDefName (CFunDef _ (CDeclr name _ _ _ _) _ _ _) = identToString $ fromJust name - -> getFunDefIdent :: CFunDef -> Ident -> getFunDefIdent (CFunDef _ (CDeclr name _ _ _ _) _ _ _) = fromJust name - -Get the parameter declarations of a function definition - -> getFunDefParams :: CFunDef -> [CDecl] -> getFunDefParams (CFunDef _ (CDeclr _ derivedDeclrs _ _ _) _ _ _) = -> let fundeclr = find isFunDeclr derivedDeclrs -> in getCFunDeclrParams (fromJust fundeclr) - -Get the return type and derived declarators for a function definition - -> getFunDefReturnMaybe :: CFunDef -> Maybe (CTypeSpec, [CDerivedDeclr]) - --- > getFunDefReturnMaybe (CFunDef specs (CDeclr _ derivedDeclrs _ _ _) _ _ _) --- > | trace ("GFDRM: " ++ show (find isTypeSpec specs)) False = undefined - -> getFunDefReturnMaybe (CFunDef specs (CDeclr _ derivedDeclrs _ _ _) _ _ _) -> | isJust (find isTypeSpec specs) = -> let (CTypeSpec tspec) = fromJust $ find isTypeSpec specs -> derivedPtrs = filter isDerivedPtr derivedDeclrs -> in Just (tspec, derivedPtrs) -> | otherwise = Nothing - --- > getFunDefReturn :: CFunDef -> (CTypeSpec, [CDerivedDeclr]) - --- > getFunDefReturn (CFunDef specs (CDeclr _ derivedDeclrs _ _ _) _ _ _) --- > | trace ("GFDR: " ++ show (find isTypeSpec specs)) False = undefined - --- > getFunDefReturn (CFunDef specs (CDeclr _ derivedDeclrs _ _ _) _ _ _) = --- > let (CTypeSpec tspec) = fromJust $ find isTypeSpec specs --- > derivedPtrs = filter isDerivedPtr derivedDeclrs --- > in (tspec, derivedPtrs) - -> isBlockingSpec :: CDeclSpec -> Bool -> isBlockingSpec (CTypeQual (CBlocking _)) = True -> isBlockingSpec _ = False - -> isPrivateSpec :: CDeclSpec -> Bool -> isPrivateSpec (CTypeQual (CPPrivate _)) = True -> isPrivateSpec _ = False - -> isSharedSpec :: CDeclSpec -> Bool -> isSharedSpec (CTypeQual (CPShared _)) = True -> isSharedSpec _ = False - -> isStdSpec :: CDeclSpec -> Bool -> isStdSpec c = (not $ isBlockingSpec c) && (not $ isPrivateSpec c) && (not $ isSharedSpec c) - -> isRemoteSpec :: CDeclSpec -> Bool -> isRemoteSpec (CTypeQual (CRemote _)) = True -> isRemoteSpec _ = False - -> getDeclSpecs :: CFunDef -> [CDeclSpec] -> getDeclSpecs (CFunDef declspecs _ _ _ _) = declspecs - -> getAllStorageSpecs :: [CDeclSpec] -> [CStorageSpec] -> getAllStorageSpecs declspecs = map getSto $ filter isSto declspecs -> where isSto (CStorageSpec s) = True -> isSto _ = False -> getSto (CStorageSpec s) = s - -> getStorageSpecs :: CFunDef -> [CStorageSpec] -> getStorageSpecs (CFunDef declspecs _ _ _ _) = getStorageSpecsFromDeclSpecs declspecs - -> getStorageSpecsFromDeclSpecs :: [CDeclSpec] -> [CStorageSpec] -> getStorageSpecsFromDeclSpecs ((CStorageSpec s):specs) -> | isStdSpec (CStorageSpec s) = s:(getStorageSpecsFromDeclSpecs specs) -> | otherwise = getStorageSpecsFromDeclSpecs specs -> getStorageSpecsFromDeclSpecs (s:specs) = getStorageSpecsFromDeclSpecs specs -> getStorageSpecsFromDeclSpecs [] = [] - -> filterDecls :: (CDeclSpec -> Bool) -> (CDeclSpec -> Bool) -> (CDecl -> CDecl) -> [CDecl] -> [CDecl] -> filterDecls anyFunc allFunc transFunc decls = map transFunc $ filter isFilterDecl decls -> where isFilterDecl (CDecl specs l ni) = any anyFunc specs && all allFunc specs - -> isStorageSpec :: CDeclSpec -> Bool -> isStorageSpec (CStorageSpec _) = True -> isStorageSpec _ = False - -> isTypeQual :: CDeclSpec -> Bool -> isTypeQual (CTypeQual _) = True -> isTypeQual _ = False - -> isStdTypeQual :: CDeclSpec -> Bool -> isStdTypeQual (CTypeQual (CRemote _)) = False -> isStdTypeQual (CTypeQual (CBlocking _)) = False -> isStdTypeQual (CTypeQual (CPPrivate _)) = False -> isStdTypeQual (CTypeQual (CPShared _)) = False -> isStdTypeQual (CTypeQual _) = True -> isStdTypeQual _ = False - -> filterStdDeclSpecs :: [CDeclSpec] -> [CDeclSpec] -> filterStdDeclSpecs declspecs = filter (\s -> isStorageSpec s || isStdTypeQual s) declspecs - -> getStdDeclSpecs :: CFunDef -> [CDeclSpec] -> getStdDeclSpecs (CFunDef declspecs _ _ _ _) = filterStdDeclSpecs declspecs - -> removeSharedSpecFromDecl :: CDecl -> CDecl -> removeSharedSpecFromDecl (CDecl specs l ni) = (CDecl rspecs l ni) -> where rspecs = filter (not . isSharedSpec) specs - -> removePrivateSpecFromDecl :: CDecl -> CDecl -> removePrivateSpecFromDecl (CDecl specs l ni) = (CDecl rspecs l ni) -> where rspecs = filter (not . isPrivateSpec) specs - -> removeRemoteSpecFromFunDef :: CExtDecl -> CExtDecl -> removeRemoteSpecFromFunDef (CFDefExt (CFunDef specs declr decls stmt ni)) = -> let newspecs = filter (not . isRemoteSpec) specs -> in (CFDefExt (CFunDef newspecs declr decls stmt ni)) - -Get the local declarations for a function definition - -> getFunLocalDeclarations :: CFunDef -> [CDecl] -> getFunLocalDeclarations (CFunDef _ _ _ (CCompound _ bitems _) _) = findFuncDecls bitems - -> getLocalDeclarations :: CStat -> [CDecl] -> getLocalDeclarations (CPBranch (CCompound _ bitems _) _) = findFuncDecls bitems -> getLocalDeclarations (CPWait (CCompound _ bitems _) _) = findFuncDecls bitems -> getLocalDeclarations _ = error "Not handled!" - -> getCallName :: CExpr -> String -> getCallName (CCall (CVar n _) _ _) = identToString n -> getCallName (CCall _ _ ni) = "fptr" -> getCallName n = error "getCallName: unknown CExpr type: " ++ (show $ pretty n) - -> getCallParams :: CExpr -> [CExpr] -> getCallParams (CCall _ p _) = p - -> getCallNodeInfo :: CExpr -> NodeInfo -> getCallNodeInfo (CCall _ _ ni) = ni - -> getBlockStmt :: CBlockItem -> Maybe CStat -> getBlockStmt (CBlockStmt stmt) = Just stmt -> getBlockStmt _ = Nothing - -> getStmtsFromBlockItems :: [CBlockItem] -> [CStat] -> getStmtsFromBlockItems bitems = [ fromJust b | b <- map getBlockStmt bitems, isJust b ] - -> getBlockDecl :: CBlockItem -> Maybe CDecl -> getBlockDecl (CBlockDecl decl) = Just decl -> getBlockDecl _ = Nothing - -> getDeclsFromBlockItems :: [CBlockItem] -> [CDecl] -> getDeclsFromBlockItems bitems = [ fromJust d | d <- map getBlockDecl bitems, isJust d ] - -> getFunStmts :: CFunDef -> [CStat] -> getFunStmts (CFunDef _ _ _ (CCompound _ bitems _) _) = getStmtsFromBlockItems bitems - -> getStmtList :: CStat -> [CStat] -> getStmtList (CCompound _ bitems _) = getStmtsFromBlockItems bitems -> getStmtList c = [c] - -> getDeclList :: CStat -> [CDecl] -> getDeclList (CCompound _ bitems _) = getDeclsFromBlockItems bitems - -> getForExprs :: CStat -> (Maybe CExpr, Maybe CExpr, Maybe CExpr) -> getForExprs (CFor (Left init) expr1 expr2 _ _) = (init, expr1, expr2) - -> isFunDeclr :: CDerivedDeclr -> Bool -> isFunDeclr (CFunDeclr _ _ _) = True -> isFunDeclr _ = False - -> isFunDecl :: CDecl -> Bool -> isFunDecl (CDecl specs drs _) = or $ map (any isFunDeclr) $ map getDerivedDeclrs drs - -> blockItemIsDecl :: CBlockItem -> Bool -> blockItemIsDecl (CBlockDecl _) = True -> blockItemIsDecl _ = False - -> isCompound :: CStat -> Bool -> isCompound (CCompound _ _ _) = True -> isCompound _ = False - -> stmtsHaveDecl :: CStat -> (Bool, NodeInfo) -> stmtsHaveDecl (CCompound i bitems ni) = (any blockItemIsDecl bitems, ni) -> stmtsHaveDecl c = (False, nodeInfo c) - -> isVoidType :: CDecl -> Bool -> isVoidType (CDecl specs declarators ni) = -> let derivedDeclrs = concatMap getDerivedDeclrs declarators -> in (not $ any isDerivedPtr derivedDeclrs) && (any isVoidTypeSpec specs) - -> isVoidReturn :: (CTypeSpec, [CDerivedDeclr]) -> Bool -> isVoidReturn (CVoidType _, []) = True -> isVoidReturn _ = False - -> isReturnStmt :: CStat -> Bool -> isReturnStmt (CReturn _ _) = True -> isReturnStmt _ = False - -> justVoid :: [CDecl] -> Bool -> justVoid decls = ((length decls) < 2) && (any isVoidType decls) - -> isBreak :: CStat -> Maybe CStat -> isBreak b@(CBreak _) = Just b -> isBreak _ = Nothing - -> isPBreak :: CStat -> Maybe CStat -> isPBreak b@(CPBreak _) = Just b -> isPBreak _ = Nothing - -> isPWaitStmt :: CStat -> Bool -> isPWaitStmt (CPWait _ _) = True -> isPWaitStmt _ = False - -> isPBranchStmt :: CStat -> Bool -> isPBranchStmt (CPBranch _ _) = True -> isPBranchStmt _ = False diff --git a/code/src/aesop/parser/CUtils.lhs b/code/src/aesop/parser/CUtils.lhs deleted file mode 100644 index 8659daf..0000000 --- a/code/src/aesop/parser/CUtils.lhs +++ /dev/null @@ -1,114 +0,0 @@ -> module CUtils (orElseM, -> takeJust, -> orElseMMaybe, -> orElseMBool, -> orElseMAdd, -> spanM, -> takeWhileM, -> spliceM, -> splitM, -> fileAndLine, -> arp_debug, -> arp_trace) where -> import Language.C -> import Language.C.System.GCC -- preprocessor used -> import System.Environment -> import Data.Typeable -> import Data.Maybe -> import Data.Either -> import Data.HashTable -> import Data.Generics -> import Data.Generics.Schemes -> import Control.Monad.State -> import qualified Data.ByteString -> import qualified Data.ByteString.Char8 -> import Control.Exception -> import Data.List -> import System.IO.Unsafe -> import Debug.Trace - -> orElseM :: (Monad m) => (a -> a -> a) -> m a -> m a -> m a -> orElseM compare x y = do mx <- x; my <- y; return $ compare mx my - -> takeJust :: Maybe a -> Maybe a -> Maybe a -> takeJust a b = if isJust a then a else b - -> orElseMMaybe :: (Monad m) => m (Maybe a) -> m (Maybe a) -> m (Maybe a) -> orElseMMaybe = orElseM takeJust - -> orElseMBool :: (Monad m) => m Bool -> m Bool -> m Bool -> orElseMBool = orElseM (||) - -> orElseMAdd :: (Monad m) => m Int -> m Int -> m Int -> orElseMAdd = orElseM (+) - -> spanM :: (Monad m) => (a -> m Bool) -> [a] -> m ([a],[a]) -> spanM _ [] = return ([], []) -> spanM p (x:xs) = do -> res <- p x -> if res then (do (ys,zs) <- spanM p xs ; return (x:ys,zs)) else (return ([], x:xs)) - -> takeWhileM :: (Monad m) => (a -> m Bool) -> [a] -> m [a] -> takeWhileM _ [] = return [] -> takeWhileM p (x:xs) = do -> res <- p x -> if res then (do as <- takeWhileM p xs ; return $ x:as) else (return []) - -Take a monad predicate and a list, for each element of the list evaluate with the predicate -and split the list at that point if the predicate returns False. So the result is a -monad that contains a list of lists. Each internal list contains elements from the previous -list up-to the failed element. For example, the list: - -[1, 2, 2, 1, 2, 3, 1, 1, 2, 3, 2, 3, 1, 2] - -and a predicate: (== 1) - -will return: - -[[1], [2, 2, 1] [2, 3, 1], [1], [2, 3, 2, 3, 1], [2]] - -> spliceM :: (Monad m) => (a -> m Bool) -> [a] -> m [[a]] -> spliceM _ [] = return [[]] -> spliceM p (x:xs) = do -> res <- p x -> l:ls <- spliceM p xs -> if res then return $ (x:l):ls else return $ [x]:l:ls - -> splitM :: (Monad m) => (a -> m Bool) -> [a] -> m [Either a [a]] -> splitM _ [] = return [] -> splitM p (a:[]) = do -> res <- p a -> if res then return [Right [a]] else return [Left a] -> splitM p (x:xs) = do -> res <- p x -> l:ls <- splitM p xs -> if res then case l of { Right as -> return $ (Right (x:as)):ls ; _ -> return $ (Right [x]):(l:ls) } else return $ (Left x):(l:ls) - -> fileAndLine :: NodeInfo -> String -> fileAndLine info = -> let file = fromJust $ fileOfNode info -> line = posRow $ posOfNode info -> in (file ++ ":" ++ (show line)) - - -DEBUGGING OUTPUT - -Toggle arp_debug_enabled for debugging output: - -> arp_debug_enabled = False - -Aesop Remote Parser debug (in IO monad): - -> arp_debug :: String -> IO () -> arp_debug s = -> if arp_debug_enabled then putStrLn ("aercc: debug: " ++ s) -> else putStr "" - -Aesop Remove Parser trace: -Like Haskell's trace(), but dependent on arp_debug_enabled - -> arp_trace :: String -> a -> a -> arp_trace string expr = -> if arp_debug_enabled -> then (trace ("aercc: debug: " ++ string) expr) -> else expr diff --git a/code/src/aesop/parser/ContextGen.lhs b/code/src/aesop/parser/ContextGen.lhs deleted file mode 100644 index cc44e3f..0000000 --- a/code/src/aesop/parser/ContextGen.lhs +++ /dev/null @@ -1,53 +0,0 @@ -> module ContextGen where -> import Language.C -> import Language.C.System.GCC -- preprocessor used -> import System.Environment -> import Data.Typeable -> import Data.Maybe -> import Data.Either -> import Data.HashTable -> import Control.Exception -> import Control.Monad.State -> import Data.List -> import Debug.Trace -> import CParse -> import CUtils -> import Walker -> import Data.Generics -> import Data.Generics.Schemes -> import Data.Tree -> import BlockingContext - -> isBlockingCall :: CExpr -> WalkerT Bool -> isBlockingCall c@(CCall v args _) = do -> res <- lookupBlocking v -> if (isJust res) then (return True) else (return False) -> isBlockingCall _ = return False - -> getBlockingExpr :: CExpr -> WalkerT [CExpr] -> getBlockingExpr c@(CCall v args _) = do -> b <- isBlockingCall c -> if b then return [c] else return [] -> getBlockingExpr _ = return [] - -> accumM :: (Monad m) => (a -> a -> a) -> m a -> m a -> m a -> accumM acc mx my = do x <- mx; y <- my; return $ acc x y - -> getAllBlockingCalls :: CFunDef -> WalkerT [CExpr] -> getAllBlockingCalls funDef = do -> everything (accumM (++)) (mkQ (return []) getBlockingExpr) funDef - -> containsBlockingCall :: CStat -> WalkerT Bool -> containsBlockingCall stmt = do -> anyBlocking <- everything orElseMBool (mkQ (return False) isBlockingCall) $ stmt -> if not anyBlocking then -> case stmt of -> (CPBranch stmts ni) -> do -> fstr <- getFilePosStr ni -> error $ fstr ++ ": The pbranch block does not contain a blocking call" -> (CPWait stmts ni) -> do -> fstr <- getFilePosStr ni -> error $ fstr ++ ": The pwait block does not contain a blocking call" -> _ -> return anyBlocking -> -> else return $ anyBlocking diff --git a/code/src/aesop/parser/Header.lhs b/code/src/aesop/parser/Header.lhs deleted file mode 100644 index 2db0710..0000000 --- a/code/src/aesop/parser/Header.lhs +++ /dev/null @@ -1,99 +0,0 @@ -> module Header where -> import Language.C -> import Language.C.Data.Position -> import Language.C.Data.Node -> import Language.C.System.GCC -- preprocessor used -> import Language.C.Data.Ident -> import System.Environment -> import System.Console.GetOpt -> import System.Exit -> import Data.Typeable -> import Data.Maybe -> import Data.Either -> import Data.HashTable -> import Data.Generics -> import Data.Generics.Schemes -> import qualified Data.ByteString -> import qualified Data.ByteString.Char8 -> import Control.Monad.State -> import Control.Exception -> import Data.List -> import CGen -> import CParse -> import CUtils -> import System.IO.Unsafe -> import Debug.Trace -> import Data.Tree -> import BlockingContext -> import Walker -> import ContextGen -> import Serialize - -> data HeaderS = HeaderS { header :: [String], trans :: Bool, pos :: Position } - -> newHeaderState :: FilePath -> IO HeaderS -> newHeaderState f = do -> s <- readFile f -> return $ HeaderS (lines s) False (position 0 f 0 0) - -> type HeaderT = StateT HeaderS IO - -> getTrans :: HeaderT Bool -> getTrans = do { h <- get ; return $ trans h } - -> setTrans :: Bool -> HeaderT () -> setTrans b = do { (HeaderS h l p) <- get ; put (HeaderS h b p) } - -> getLastPos :: HeaderT Position -> getLastPos = do { h <- get ; return $ pos h } - -> setLastPos :: Position -> HeaderT () -> setLastPos np = do { (HeaderS h l p) <- get ; put (HeaderS h l np) } - -> getHeaderLines :: Int -> HeaderT [String] -> getHeaderLines i = do -> (HeaderS h l p) <- get -> let (n, rest) = splitAt i $ h -> put (HeaderS rest l p) -> return n - -> getRemainingLines :: HeaderT [String] -> getRemainingLines = do -> (HeaderS h l p) <- get -> return h - -> skipBytes :: Int -> HeaderT () -> skipBytes i = do -> (HeaderS h l p) <- get -> let (n, rest) = splitAt i $ unlines h -> put (HeaderS (lines rest) l p) - -> outputHeader :: FilePath -> FilePath -> [([CExtDecl], CExtDecl)] -> IO () -> outputHeader hf outf pairs = do -> hs <- newHeaderState hf -> runStateT (outputPairs hf outf pairs) $ hs -> return () - -> outputPairs :: FilePath -> FilePath -> [([CExtDecl], CExtDecl)] -> HeaderT () -> outputPairs hf outf pairs = do -> mapM_ (outputPair hf outf) pairs -> l <- getRemainingLines -> liftIO $ appendFile outf $ unlines l - -> outputPair :: FilePath -> FilePath -> ([CExtDecl], CExtDecl) -> HeaderT () -> outputPair hf outf ([], c) | isInFile c hf = return () -> outputPair hf outf (ds, c) | isInFile c hf = do -> lastPos <- getLastPos -> let currentPos = posOfNode $ nodeInfo c -> n = (posRow currentPos) - (posRow lastPos) - 1 -> ltp = fst $ getLastTokenPos $ nodeInfo c -> hl <- getHeaderLines n -> getHeaderLines $ (posRow $ ltp) - (posRow currentPos) + 1 -> liftIO $ appendFile outf $ unlines hl -> liftIO $ appendFile outf $ unlines $ map (show . pretty) ds -> setLastPos $ ltp -> outputPair hf outf (ds, c) = do -> return () - -> isInFile :: CExtDecl -> String -> Bool -> isInFile c f = (posFile $ posOfNode $ nodeInfo c) == f diff --git a/code/src/aesop/parser/Serialize.hs b/code/src/aesop/parser/Serialize.hs deleted file mode 100644 index 015a473..0000000 --- a/code/src/aesop/parser/Serialize.hs +++ /dev/null @@ -1,139 +0,0 @@ -{-# LANGUAGE TypeSynonymInstances #-} - -module Serialize ( - Serialize (..) -) where - -import Text.PrettyPrint.HughesPJ - -import Language.C.Pretty -import Language.C.Data -import Language.C.Syntax - -class (Pretty s) => Serialize s where - - serialize :: s -> Doc - serializePrec :: Int -> s -> Doc - - serialize = pretty - serializePrec _ = serialize - -szLine :: NodeInfo -> Doc -szLine ni = text "#" <+> int (posRow $ posOfNode ni) <+> (doubleQuotes $ text (posFile $ posOfNode ni)) - --- serialize optional chunk -maybeP :: (p -> Doc) -> Maybe p -> Doc -maybeP = maybe empty - --- serialize when flag is true -ifP :: Bool -> Doc -> Doc -ifP flag doc = if flag then doc else empty - --- serialize _optional_ list, i.e. [] ~ Nothing and (x:xs) ~ Just (x:xs) -mlistP :: ([p] -> Doc) -> [p] -> Doc -mlistP pp xs = maybeP pp (if null xs then Nothing else Just xs) - --- serialize identifier -identP :: Ident -> Doc -identP = text . identToString - --- pretty print attribute annotations -attrlistP :: [CAttr] -> Doc -attrlistP [] = empty -attrlistP attrs = text "__attribute__" <> parens (parens (hcat . punctuate comma . map pretty $ attrs)) - --- analogous to showParen -parenPrec :: Int -> Int -> Doc -> Doc -parenPrec prec prec2 t = if prec <= prec2 then t else parens t - --- indent a chunk of code -ii :: Doc -> Doc -ii = nest 4 - --- Serialize instances -instance Serialize CTranslUnit where - serialize c@(CTranslUnit edecls ni) = szLine ni $+$ vcat (map serialize edecls) - --- TODO: Check need of __extension__ -instance Serialize CExtDecl where - serialize (CDeclExt decl) = (szLine $ nodeInfo decl) $+$ serialize decl <> semi - serialize (CFDefExt fund) = (szLine $ nodeInfo fund) $+$ serialize fund - serialize (CAsmExt asmStmt _) = text "asm" <> parens (pretty asmStmt) <> semi - -instance Serialize CFunDef where - serialize (CFunDef declspecs declr decls stat ni) = -- Example: - szLine ni - $+$ hsep (map pretty declspecs) -- __attribute__((noreturn)) static long - <+> pretty declr -- foo(b) - $+$ (ii . vcat . map (<> semi) . map pretty) decls -- register long b; - $+$ serializePrec (-1) stat -- { ... - -- } - -instance Serialize CStat where - serialize (CLabel ident stat cattrs ni) = szLine ni $+$ identP ident <> text ":" <+> attrlistP cattrs $$ serialize stat - serialize (CCase expr stat ni) = - szLine ni $+$ text "case" <+> pretty expr <> text ":" $$ serialize stat - serialize (CCases expr1 expr2 stat ni) = - szLine ni $+$ text "case" <+> pretty expr1 <+> text "..." - <+> pretty expr2 <> text ":" $$ serialize stat - serialize (CDefault stat ni) = szLine ni $+$ text "default:" $$ serialize stat - serialize (CExpr expr ni) = szLine ni $+$ (ii $ maybeP pretty expr <> semi) - serialize c@(CCompound _ _ _) = serializePrec 0 c - serialize (CIf expr stat estat ni) = - szLine ni $+$ - (ii $ text "if" <+> text "(" <> pretty expr <> text ")") - $+$ serializePrec (-1) stat - $+$ maybeP szElse estat - where - szElse (CIf else_if_expr else_if_stat else_stat ni) = - szLine ni $+$ - text "else if" <+> text "(" <> pretty else_if_expr <> text ")" - $+$ serializePrec (-1) else_if_stat - $+$ maybeP szElse else_stat - szElse else_stmt = - text "else" $+$ serializePrec (-1) else_stmt - - serialize (CSwitch expr stat ni) = - szLine ni $+$ - (ii $ text "switch" <+> text "(" <> pretty expr <> text ")" - $+$ serializePrec (-1) stat) - serialize (CWhile expr stat False ni) = - szLine ni $+$ - (ii $ text "while" <+> text "(" <> pretty expr <> text ")" - $+$ serializePrec (-1) stat) - serialize (CWhile expr stat True ni) = - szLine ni $+$ - (ii $ text "do" $+$ serializePrec (-1) stat - $+$ text "while" <+> text "(" <> pretty expr <> text ");") - serialize (CFor for_init cond step stat ni) = - szLine ni $+$ - (ii $ text "for" <+> text "(" - <> either (maybeP pretty) pretty for_init <> semi - <+> maybeP pretty cond <> semi - <+> maybeP pretty step <> text ")" $+$ serializePrec (-1) stat) - serialize (CGoto ident ni) = szLine ni $+$ (ii $ text "goto" <+> identP ident <> semi) - serialize (CGotoPtr expr ni) = szLine ni $+$ (ii $ text "goto" <+> text "*" <+> prettyPrec 30 expr <> semi) - serialize (CCont ni) = szLine ni $+$ (ii $ text "continue" <> semi) - serialize (CBreak ni) = szLine ni $+$ (ii $ text "break" <> semi) - serialize (CReturn Nothing ni) = szLine ni $+$ (ii $ text "return" <> semi) - serialize (CReturn (Just e) ni) = szLine ni $+$ (ii $ text "return" <+> pretty e <> semi) - serialize (CAsm asmStmt ni) = szLine ni $+$ pretty asmStmt - serialize (CPWait stmt ni) = - szLine ni $+$ - (ii $ text "pwait" $+$ prettyPrec (-1) stmt) - serialize (CPBranch stmt ni) = szLine ni $+$ (ii $ text "pbranch" $+$ prettyPrec (-1) stmt) - serialize (CPBreak ni) = szLine ni $+$ (ii $ text "pbreak" <> semi) - serializePrec p (CCompound localLabels bis ni) = - let inner = text "{" $+$ mlistP ppLblDecls localLabels $+$ vcat (map serialize bis) $+$ text "}" - in if p == -1 then inner else ii inner - where ppLblDecls = vcat . map (\l -> text "__label__" <+> identP l <+> semi) - serializePrec _ p = serialize p - --- TODO: Check need of __extension__ -instance Serialize CBlockItem where - serialize (CBlockStmt stat) = serialize stat - serialize (CBlockDecl decl) = ii $ serialize decl <> semi - serialize (CNestedFunDef fundef) = ii $ pretty fundef - -instance Serialize CDecl where - serialize c@(CDecl specs divs ni) = szLine ni $+$ pretty c diff --git a/code/src/aesop/parser/Walker.lhs b/code/src/aesop/parser/Walker.lhs deleted file mode 100644 index d195385..0000000 --- a/code/src/aesop/parser/Walker.lhs +++ /dev/null @@ -1,352 +0,0 @@ -> module Walker where -> import Language.C -> import Language.C.System.GCC -- preprocessor used -> import Language.C.Pretty -> import Language.C.Data.Ident -> import System.Environment -> import Data.Typeable -> import Data.Maybe -> import Data.Either -> import Data.HashTable -> import Control.Exception -> import Data.Generics -> import Data.Generics.Schemes -> import Data.List -> import Debug.Trace -> import Control.Monad.State -> import BlockingContext -> import CParse -> import CGen -> import Control.Monad.Error - -Registry for blocking operations. Once we see a function declaration -with the __blocking specifier, we add it to the registry. - -> type FunDecl = (Ident, ReturnType, [CDecl]) - -> funDeclName :: FunDecl -> Ident -> funDeclName (n, _, _) = n - -> funDeclReturnType :: FunDecl -> ReturnType -> funDeclReturnType (_, r, _) = r - -> funDeclParams :: FunDecl -> [CDecl] -> funDeclParams (_, _, p) = p - -The FPType is either a function or a struct. The FPFun is the -function pointer, declarated within a struct or simply declarated locally. - -The struct (FPStruct) is a struct with a function pointer as one of its fields. - -> data FPType = FPFun { name :: Ident , ftype :: FunDecl } -> | FPStruct { name :: Ident , field :: Ident, inner :: FPType } - -> instance Show FPType where -> show (FPFun n _) = "Fun: " ++ (show n) -> show (FPStruct n f t) = "Struct: " ++ (show n) ++ ":" ++ (show f) ++ "[" ++ (show t) ++ "]" - -> isFPStruct :: FPType -> Bool -> isFPStruct (FPStruct _ _ _) = True -> isFPStruct _ = False - -> getInner :: FPType -> Maybe FPType -> getInner (FPStruct _ _ t) = Just t -> getInner _ = Nothing - -> isFPFun :: FPType -> Bool -> isFPFun (FPFun _ _) = True -> isFPFun _ = False - -> getFunDecl :: FPType -> Maybe FunDecl -> getFunDecl (FPFun _ f) = Just f -> getFunDecl _ = Nothing - -> type FPTypeRegistry = [FPType] - -> type VarMap = HashTable Ident CDecl - -> data VarRegistry = VarRegistry { locals :: VarMap , globals :: VarMap } - -> newVarRegistry :: IO VarRegistry -> newVarRegistry = do -> l <- Data.HashTable.new (==) (Data.HashTable.hashInt . hashIdent) -> g <- Data.HashTable.new (==) (Data.HashTable.hashInt . hashIdent) -> return $ VarRegistry l g - -> resetLocalVarsInRegistry :: VarRegistry -> IO VarRegistry -> resetLocalVarsInRegistry v = do -> l <- Data.HashTable.new (==) (Data.HashTable.hashInt . hashIdent) -> return $ VarRegistry l (globals v) - -> insertGlobalDeclsToVarRegistry :: [CDecl] -> VarRegistry -> IO () -> insertGlobalDeclsToVarRegistry decls v = do -> let alldecls = concatMap splitDecls decls -> mapM_ (\d -> Data.HashTable.insert (globals v) (getCDeclName $ d) d) alldecls - -> insertLocalDeclsToVarRegistry :: [CDecl] -> VarRegistry -> IO () -> insertLocalDeclsToVarRegistry decls v = do -> let alldecls = concatMap splitDecls decls -> insertD d = Data.HashTable.insert (locals v) (getCDeclName $ d) d -> mapM_ insertD alldecls - -> lookupInVarRegistry :: Ident -> VarRegistry -> IO (Maybe CDecl) -> lookupInVarRegistry i v = do -> res <- Data.HashTable.lookup (locals v) i -> if isJust res -> then return res -> else Data.HashTable.lookup (globals v) i - -The monad transformer that we thread through our functions - -The Walker is the state that we pass around our functions as we walk the AST. It primarily holds -the blocking call information, but is actually a tuple of: -filename, blocking call registry, and blocking function pointer registry - -> data Walker = Walker { -> debug :: Bool, -> filename :: String, -> compiler :: String, -> fpTypeReg :: FPTypeRegistry, -> fpTypeLocalReg :: FPTypeRegistry, -> varReg :: VarRegistry, -> blockingParser :: Maybe MacroParser -> } - -> newWalkerState :: Bool -> -> String -> -> FilePath -> -> FilePath -> -> [String] -> -> [Ident] -> -> IO Walker - -> newWalkerState debug fname compiler macroHeader gccopts tdIdents = do -> varReg <- newVarRegistry -> bp <- mkParser compiler macroHeader gccopts -> let nbp = addTypeIdents tdIdents bp -> return $ Walker debug fname compiler [] [] varReg (Just nbp) - -> setBlockingParser :: MacroParser -> WalkerT () -> setBlockingParser b = do -> w <- get -> put $ w { blockingParser = Just b } - -> getBlockingParser :: WalkerT MacroParser -> getBlockingParser = do -> w <- get -> let bp = blockingParser w -> return $ assert (isJust bp) (fromJust bp) - -> type WalkerT = StateT Walker IO - -> wdebug :: String -> WalkerT () -> wdebug msg = do -> w <- get -> when (debug w) $ putStrLnW msg -> return () - -> getFileName :: WalkerT String -> getFileName = do -> w <- get -> return $ filename w - -> getFilePosStr :: NodeInfo -> WalkerT (String) -> getFilePosStr (NodeInfo p _ _) = do -> fname <- getFileName -> return (fname ++ ":" ++ (show $ posRow p) ++ ":" ++ (show $ posColumn p)) - -> clearLocalFunPtrRegistry :: WalkerT () -> clearLocalFunPtrRegistry = do -> w <- get -> put $ w { fpTypeLocalReg = [] } - -> resetLocals :: WalkerT () -> resetLocals = do -> w <- get -> newvr <- liftIO $ resetLocalVarsInRegistry $ varReg w -> put $ w { varReg = newvr } - -> addLocals :: [CDecl] -> WalkerT () -> addLocals ds = do -> w <- get -> liftIO $ insertLocalDeclsToVarRegistry ds $ varReg w - -> addGlobals :: [CDecl] -> WalkerT () -> addGlobals ds = do -> w <- get -> liftIO $ insertGlobalDeclsToVarRegistry ds $ varReg w - -Get the chain of variable references starting -at the outermost. i.e. - -struct a a1; -struct b b1; -typedef struct c ct; -ct c1; -struct d d1; - -a1->b1->c1->d1 - -Turns into: - -(a, [b1, c1, d1]) - -> lookupVar :: CExpr -> WalkerT (Maybe [Ident]) -> lookupVar e@(CVar name _) = do -> wdebug $ "[lookupVar]: CVar: " ++ (show name) -> w <- get -> d <- liftIO $ lookupInVarRegistry name $ varReg w -> if isNothing d -> then return Nothing -> else do -> let t = getTypeSpecFromDecl $ fromJust d -> r = case t of -> (CTypeDef tname _) -> Just [tname] -> (CSUType (CStruct _ (Just tname) _ _ _) _) -> Just [tname] -> _ -> Nothing -> if isFunDecl $ fromJust d -> then return $ Just [getCDeclName $ fromJust d] -> else if isJust r -> then return r -> else return Nothing - -> lookupVar (CMember expr name _ _) = do -> wdebug $ "[lookupVar]: CMember: " ++ (show name) -> w <- get -> inner <- lookupVar expr -> if isNothing inner -> then do -> return Nothing -> else return $ Just $ (fromJust inner) ++ [name] - -> lookupVar e = do -> return Nothing - -> insertFP :: FPType -> WalkerT () -> insertFP fptype = do -> w <- get -> put $ w { fpTypeReg = fptype : (fpTypeReg w) } - -Register a blocking function pointer. - -> registerBlocking :: FunDecl -> WalkerT () -> registerBlocking fd = do -> let (n, (rt, rd), p) = fd -> insertFP $ FPFun n fd -> let d = mkIdentFunDecl n rt rd p -> addGlobals [d] - -Register a blocking struct based on the struct type (structName), the field -with the function pointer, and type of the function pointer (function signature). - -> registerBlockingStruct :: Ident -> Ident -> FunDecl -> WalkerT () -> registerBlockingStruct structName fieldName funDecl = do -> let (n, (rt, rd), p) = funDecl -> insertFP $ FPStruct structName fieldName (FPFun n funDecl) - -We have to deal with structs that have members that are structs that have blocking -function pointers. I.e: - -struct inner -{ - __blocking void (*blockingfun)(void); -}; - -struct outer -{ - struct inner i; -}; - -So we need a way to lookup if a field within a struct is a blocking struct, and -if so, register the outer struct too. - -> lookupAndRegisterBlockingStruct :: Ident -> Ident -> Ident -> WalkerT () -> lookupAndRegisterBlockingStruct structName fieldName fieldType = do -> w <- get -> let reg = fpTypeReg w -> matching = filter (\f -> (isFPStruct f && ((name f) == fieldType))) reg -> if not $ null matching -> then do -> mapM_ (\m -> insertFP $ FPStruct structName fieldName m) matching -> else return () - -> lookupAndRegisterBlockingTypedef :: Ident -> Ident -> WalkerT () -> lookupAndRegisterBlockingTypedef structName typeName = do -> w <- get -> let reg = fpTypeReg w -> matching = filter (\f -> (isFPStruct f && ((name f) == structName))) reg -> if not $ null matching -> then do -> -- when ((length matching) /= 1) $ do -> -- putStrLnW $ identToString structName -> -- mapM_ (putStrLnW . show) matching -> -- assert ((length matching) == 1) return () -> let mkFP (FPStruct s f fpt) = FPStruct typeName f fpt -> mapM_ (insertFP . mkFP) matching -> else return () - -> registerLocalBlocking :: FunDecl -> WalkerT () -> registerLocalBlocking fd = do -> w <- get -> let (n, _, _) = fd -> put $ w { fpTypeLocalReg = (FPFun n fd) : (fpTypeLocalReg w) } - -> getAllBlocking :: WalkerT [FPType] -> getAllBlocking = do -> w <- get -> let bs = fpTypeReg w -> bls = fpTypeLocalReg w -> return $ bs ++ bls - -> showAllBlocking :: WalkerT () -> showAllBlocking = do -> all <- getAllBlocking -> putStrLnW $ unlines $ map show all - -> tr :: [String] -> FPType -> String -> a -> a -> tr ss st a = trace $ a ++ "::" ++ (show ss) ++ "::" ++ (show st) - -> matchFP :: [Ident] -> FPType -> Maybe FunDecl -> matchFP ss@(s:p:seqs) st@(FPStruct n f t) -> | s == n && p == f && null seqs && isFPFun t = getFunDecl t -> | s == n && p == f = matchiFP seqs (fromJust (getInner t)) -> | otherwise = Nothing - -> matchFP s f = matchiFP s f - -> matchiFP :: [Ident] -> FPType -> Maybe FunDecl -> matchiFP [] _ = Nothing - -> matchiFP ss@(s:[]) st@(FPFun n t) = if s == n then Just t else Nothing -> matchiFP ss@(s:seqs) st@(FPFun n t) = Nothing -> matchiFP ss@(s:seqs) st@(FPStruct n f t) = -> if s == f then matchFP seqs t else Nothing - -> lookupBlocking :: CExpr -> WalkerT (Maybe FunDecl) -> lookupBlocking (CCall e _ _) = do -> lookupBlocking e - -> lookupBlocking c = do -> wdebug $ "[lookupBlocking]: " ++ (show $ pretty c) -> v <- lookupVar c -> if isNothing v -> then do -> wdebug $ "[lookupBlocking]: returning Nothing" -> return Nothing -> else do -> all <- getAllBlocking -> let fs = mapMaybe (matchFP $ fromJust v) $ all -> if null fs then do -> wdebug $ "[lookupBlocking]: returning Nothing" -> return Nothing -> else do -> wdebug $ "[lookupBlocking]: returning " ++ (identToString $ funDeclName $ head fs) -> return $ Just $ head fs - -Utility lift the print function into the WalkerT monad transformer - -> putStrLnW :: String -> WalkerT () -> putStrLnW = liftIO . putStrLn - -> putStrW :: String -> WalkerT () -> putStrW = liftIO . putStr diff --git a/code/src/aesop/parser/ae-remote-parser.lhs b/code/src/aesop/parser/ae-remote-parser.lhs deleted file mode 100644 index febd64c..0000000 --- a/code/src/aesop/parser/ae-remote-parser.lhs +++ /dev/null @@ -1,1104 +0,0 @@ -> module RemoteParser where -> import Language.C -> import Language.C.Data.Position -> import Language.C.System.GCC -- preprocessor used -> import Language.C.Data.Ident -> import System.Environment -> import System.Console.GetOpt -> import System.Exit -> import Data.Typeable -> import Data.Maybe -> import Data.Either -> import Data.HashTable -> import Data.Generics -> import Data.Generics.Schemes -> import qualified Data.ByteString -> import qualified Data.ByteString.Char8 -> import Control.Monad.State -> import Control.Exception -> import Data.List -> import CGen -> import CParse -> import CUtils -> import System.IO -> import System.IO.Unsafe -> import Debug.Trace -> import Data.Tree -> import Serialize -> import CParse -> import CGen -> import Header -> import Text.Regex.PCRE -> import Data.Array -> import System.Directory - -Translates an aer.i file to an aer.r file - -Each type in the registry is the name and a set of truth values each function -required: (encode, decode, encode_size, init, destroy) -Once one of those function declarations is found, the appropriate boolean value -is set to true. - -Note that stderr is visible long before putStrLn! - -Each transformation is processed by a call to CPP -The call to CPP uses a constructed, unique -include file -The -include file only contains definitions from ae-remote-parser.h and the - things it includes - anything else will not be found - -> replace y z [] = [] -> replace y z (x:xs) -> |x==y = z:replace y z xs -> |otherwise = x:replace y z xs - -> data RemoteType = RemoteType { -> encode :: Bool, -> decode :: Bool, -> encodeSize :: Bool, -> initNull :: Bool, -> destroy :: Bool, -> copy :: Bool -> } - -> remoteTypeAllFalse = RemoteType False False False False False False -> remoteTypeAllTrue = RemoteType True True True True True True - -> type RemoteTypeRegistry = HashTable String RemoteType - -> newRemoteTypeRegistry :: IO RemoteTypeRegistry -> newRemoteTypeRegistry = Data.HashTable.new (==) Data.HashTable.hashString - -> data Remote = Remote { -> filename :: FilePath, -> typeReg :: RemoteTypeRegistry, -> decls :: [CExtDecl], -> funs :: [(String, (CTypeSpec, [CDerivedDeclr]), [CDecl])], -> compiler :: String, -> remoteParser :: Maybe MacroParser -> } - -> getRemotes :: RemoteT [(String, (CTypeSpec, [CDerivedDeclr]), [CDecl])] -> getRemotes = do -> r <- get -> return $ funs r - -> getRemoteNames :: RemoteT [String] -> getRemoteNames = do -> ts <- getRemotes -> return $ map (\((,,) f _ _) -> f) ts - -> type RemoteT = StateT Remote IO - -> invalid :: String -> NodeInfo -> RemoteT () -> invalid msg ni = do -> fstr <- getFilePosStr ni -> error (fstr ++ ": Invalid aesop usage: " ++ msg) - -> registerRemoteFun :: String -> Maybe (CTypeSpec, [CDerivedDeclr]) -> [CDecl] -> RemoteT () -> registerRemoteFun name retType params = do -> r <- get -> put $ r { funs = (name, fromJust retType, params) : (funs r) } - -> registerEncodeFun :: String -> RemoteT () -> registerEncodeFun e = do -> r <- get -> res <- liftIO $ Data.HashTable.lookup (typeReg r) e -> if isJust res -> then do -> let rtype = fromJust res -> liftIO $ Data.HashTable.insert (typeReg r) e $ rtype { encode = True } -> else -> liftIO $ Data.HashTable.insert (typeReg r) e $ remoteTypeAllFalse { encode = True } -> return () - -> registerDecodeFun :: String -> RemoteT () -> registerDecodeFun d = do -> r <- get -> res <- liftIO $ Data.HashTable.lookup (typeReg r) d -> if isJust res -> then do -> let rtype = fromJust res -> liftIO $ Data.HashTable.insert (typeReg r) d $ rtype { decode = True } -> else -> liftIO $ Data.HashTable.insert (typeReg r) d $ remoteTypeAllFalse { decode = True } -> return () - -> registerEncodeSizeFun :: String -> RemoteT () -> registerEncodeSizeFun d = do -> r <- get -> res <- liftIO $ Data.HashTable.lookup (typeReg r) d -> if isJust res -> then do -> let rtype = fromJust res -> liftIO $ Data.HashTable.insert (typeReg r) d $ rtype { encodeSize = True } -> else -> liftIO $ Data.HashTable.insert (typeReg r) d $ remoteTypeAllFalse { encodeSize = True } -> return () - -> registerCopyFun :: String -> RemoteT () -> registerCopyFun d = do -> r <- get -> res <- liftIO $ Data.HashTable.lookup (typeReg r) d -> if isJust res -> then do -> let rtype = fromJust res -> liftIO $ Data.HashTable.insert (typeReg r) d $ rtype { copy = True } -> else -> liftIO $ Data.HashTable.insert (typeReg r) d $ remoteTypeAllFalse { copy = True } -> return () - -> registerInitNullFun :: String -> RemoteT () -> registerInitNullFun d = do -> r <- get -> res <- liftIO $ Data.HashTable.lookup (typeReg r) d -> if isJust res -> then do -> let rtype = fromJust res -> liftIO $ Data.HashTable.insert (typeReg r) d $ rtype { initNull = True } -> else -> liftIO $ Data.HashTable.insert (typeReg r) d $ remoteTypeAllFalse { initNull = True } -> return () - -> registerDestroyFun :: String -> RemoteT () -> registerDestroyFun d = do -> r <- get -> res <- liftIO $ Data.HashTable.lookup (typeReg r) d -> if isJust res -> then do -> let rtype = fromJust res -> liftIO $ Data.HashTable.insert (typeReg r) d $ rtype { destroy = True } -> else -> liftIO $ Data.HashTable.insert (typeReg r) d $ remoteTypeAllFalse { destroy = True } -> return () - -> isFullEncoding :: String -> RemoteT Bool -> isFullEncoding d = do -> r <- get -> res <- liftIO $ Data.HashTable.lookup (typeReg r) d -> let rtype = fromJust res -> if isJust res && encode rtype && decode rtype && encodeSize rtype -> && initNull rtype && copy rtype && destroy rtype -> then return True -> else return False - -> registerFullEncoding :: String -> NodeInfo -> RemoteT () -> registerFullEncoding s ni = do -> r <- get -> res <- liftIO $ Data.HashTable.lookup (typeReg r) s -> when (isJust res) $ invalid (s ++ " already has encoding functions defined.\n") ni -> liftIO $ Data.HashTable.insert (typeReg r) s remoteTypeAllTrue - -> addRemoteDecl :: CExtDecl -> NodeInfo -> RemoteT () -> addRemoteDecl d ni = do -> r <- get -> put $ r { decls = d : (decls r) } - -> newRemoteState :: String -> String -> FilePath -> [String] -> IO Remote -> newRemoteState fname compiler macroHeader gccopts = do -> r <- Data.HashTable.new (==) Data.HashTable.hashString -> bp <- mkParser compiler macroHeader gccopts -> return $ Remote fname r [] [] compiler (Just bp) - -> getRemoteTypeName :: CTypeSpec -> Maybe String -> getRemoteTypeName (CSUType (CStruct CStructTag (Just n) _ _ _) _) = Just $ "struct_" ++ (identToString n) -> getRemoteTypeName (CSUType (CStruct CUnionTag (Just n) _ _ _) _) = Just $ "union_" ++ (identToString n) -> getRemoteTypeName (CEnumType (CEnum (Just n) _ _ _) _) = Just $ "enum_" ++ (identToString n) -> getRemoteTypeName (CTypeDef n _) = Just $ identToString n -> getRemoteTypeName _ = Nothing - -> mkRemoteDecl :: CDecl -> [CDerivedDeclr] -> String -> Maybe CDecl -> mkRemoteDecl d@(CDecl specs inits ni) ders v = -> let s = getTypeSpec specs -> in case s of -> (CSUType (CStruct CStructTag name _ _ ni2) ni3) -> Just $ mkCDecl (CSUType (CStruct CStructTag name Nothing [] ni2) ni3) ders v ni -> (CSUType (CStruct CUnionTag name _ _ ni2) ni3) -> Just $ mkCDecl (CSUType (CStruct CUnionTag name Nothing [] ni2) ni3) ders v ni -> (CEnumType (CEnum name _ _ ni2) ni3) -> Just $ mkCDecl (CEnumType (CEnum name Nothing [] ni2) ni3) ders v ni -> (CTypeDef name ni2) -> Just $ mkCDecl (CTypeDef name ni2) ders v ni -> _ -> Nothing - -> uncurry3 :: (a -> b -> c -> d) -> ((a, b, c) -> d) -> uncurry3 f (a, b, c) = f a b c - -> getFieldInfo :: CDecl -> RemoteT (String, String, Bool) -> getFieldInfo d@(CDecl specs declrs ni) = do -> let t = getTypeSpec specs -> ds = getCDeclrsFromList declrs -> ptrs = if null ds then [] else getDerivedPtrs $ head ds -> tname = getRemoteTypeName t -> isPtr = not $ null ptrs -> varName = getVarName d -> when (length ptrs > 1) $ invalid ("parameter '" ++ (show $ pretty d) ++ "' has too many pointers, only one pointer supported") ni -> when (isNothing tname) $ invalid ("parameter '" ++ (show $ pretty d) ++ "' does not have a known encoding type") ni -> return (fromJust tname, varName, isPtr) - -typeof(expr) - -CTypeOfType CDecl NodeInfo - -> mkDeclsFromRemote :: String -> [String] -> NodeInfo -> RemoteT [CDecl] -> mkDeclsFromRemote macro params ni = (arp_trace "mkDeclsFromRemote") $ do -> r <- get -> let mp = assert (isJust $ remoteParser r) (fromJust $ remoteParser r) -> return $ mkDeclsFromCPPMacro mp ni macro params - -> mkStmtFromRemote :: String -> [String] -> NodeInfo -> RemoteT [CStat] -> mkStmtFromRemote macro params ni = (arp_trace "mkStmtFromRemote") $ do -> r <- get -> let mp = assert (isJust $ remoteParser r) (fromJust $ remoteParser r) -> return $ mkStmtsFromCPPMacro mp ni macro params - -> mkFunDefFromRemote :: String -> [CStat] -> String -> [String] -> NodeInfo -> RemoteT [CExtDecl] -> mkFunDefFromRemote name block macro params ni = (arp_trace "mkFunDefFromRemote") $ do -> r <- get -> let mp = assert (isJust $ remoteParser r) (fromJust $ remoteParser r) -> return $ mkFunDefFromCPPMacro mp ni macro params name block - -> mkEncodeBlock :: NodeInfo -> String -> String -> Bool -> RemoteT [CStat] -> mkEncodeBlock ni typeName fieldName isPtr = do -> let ptrParam = if isPtr then "" else "&" -> mkStmtFromRemote "AER_MK_ENCODE_TYPE" [typeName, fieldName, ptrParam] ni - -> mkEncodeStmts :: CDecl -> [CDecl] -> NodeInfo -> RemoteT CStat -> mkEncodeStmts stype fields ni = do -> fieldsInfo <- mapM getFieldInfo fields -> let anonSType = (arp_trace "mkAnonFromDecl" ) mkAnonFromDecl stype -> encodeBlocks <- liftM concat $ sequence $ map (uncurry3 $ mkEncodeBlock ni) fieldsInfo -> encodeDecls <- mkDeclsFromRemote "AER_MK_ENCODE_DECLS" [show $ pretty anonSType] ni -> let typeStr = show $ pretty anonSType -> canon ' ' = '_' -> canon '*' = 'p' -> canon '{' = '_'; -> canon '}' = '_'; -> canon a = a -> canonTypeStr = map canon typeStr -> startStmts <- mkStmtFromRemote "AER_MK_ENCODE_STMTS_START" [typeStr, canonTypeStr] ni -> endStmts <- mkStmtFromRemote "AER_MK_ENCODE_STMTS_END" [] ni -> return $ mkCompoundWithDecls Nothing encodeDecls (startStmts ++ encodeBlocks ++ endStmts) ni - -> mkEncodeFun :: CDecl -> String -> [CDecl] -> NodeInfo -> RemoteT CExtDecl -> mkEncodeFun decl sname fields ni = do -> let tparam = mkRemoteDecl decl [(CPtrDeclr [] ni)] "x" -> when (isNothing tparam) $ invalid "not a known encoding type" ni -> stmts <- (arp_trace "mkEncodeStmts") mkEncodeStmts (fromJust tparam) fields ni -> return $ mkFunDef ((CTypeDef (newIdent "triton_ret_t" ni) ni), []) -> --[(CStorageSpec (CStatic ni)), (CTypeQual (CInlineQual ni))] -> [] -> ("aer_encode_" ++ sname) -> [mkCDecl (CTypeDef (newIdent "triton_buffer_t" ni) ni) [CPtrDeclr [] ni] "buf" ni, -> mkConstCDecl (CCharType ni) [CPtrDeclr [] ni] "varname" ni, -> mkCDecl (CVoidType ni) [CPtrDeclr [] ni] "vx" ni] -> stmts - -> mkDecodeBlock :: NodeInfo -> String -> String -> Bool -> RemoteT [CStat] -> mkDecodeBlock ni typeName fieldName isPtr = do -> let ptrParam = if isPtr then "" else "&" -> mkStmtFromRemote "AER_MK_DECODE_TYPE" [typeName, fieldName, ptrParam] ni - -> mkDecodeStmts :: CDecl -> [CDecl] -> NodeInfo -> RemoteT CStat -> mkDecodeStmts stype fields ni = do -> fieldsInfo <- mapM getFieldInfo fields -> let anonSType = mkAnonFromDecl stype -> decodeBlocks <- liftM concat $ sequence $ map (uncurry3 $ mkDecodeBlock ni) fieldsInfo -> decodeDecls <- mkDeclsFromRemote "AER_MK_DECODE_DECLS" [show $ pretty anonSType] ni -> let typeStr = show $ pretty anonSType -> canon ' ' = '_' -> canon '*' = 'p' -> canon '{' = '_'; -> canon '}' = '_'; -> canon a = a -> canonTypeStr = map canon typeStr -> startStmts <- mkStmtFromRemote "AER_MK_DECODE_STMTS_START" [typeStr, canonTypeStr] ni -> endStmts <- mkStmtFromRemote "AER_MK_DECODE_STMTS_END" [] ni -> return $ mkCompoundWithDecls Nothing decodeDecls (startStmts ++ decodeBlocks ++ endStmts) ni - -> mkDecodeFun :: CDecl -> String -> [CDecl] -> NodeInfo -> RemoteT CExtDecl -> mkDecodeFun decl sname fields ni = do -> let tparam = mkRemoteDecl decl [(CPtrDeclr [] ni)] "x" -> when (isNothing tparam) $ invalid "not a known encoding type" ni -> stmts <- mkDecodeStmts (fromJust tparam) fields ni -> return $ mkFunDef ((CTypeDef (newIdent "triton_ret_t" ni) ni), []) -> --[(CStorageSpec (CStatic ni)), (CTypeQual (CInlineQual ni))] -> [] -> ("aer_decode_" ++ sname) -> [mkCDecl (CTypeDef (newIdent "triton_buffer_t" ni) ni) [CPtrDeclr [] ni] "buf" ni, -> mkCDecl (CCharType ni) [CPtrDeclr [] ni, CPtrDeclr [] ni] "varname" ni, -> mkCDecl (CVoidType ni) [CPtrDeclr [] ni] "vx" ni] -> stmts - -> mkSizeBlock :: NodeInfo -> String -> String -> Bool -> RemoteT [CStat] -> mkSizeBlock ni typeName fieldName isPtr = do -> let ptrParam = if isPtr then "" else "&" -> mkStmtFromRemote "AER_MK_SIZE_TYPE" [typeName, fieldName, ptrParam] ni - -> mkSizeStmts :: CDecl -> [CDecl] -> NodeInfo -> RemoteT CStat -> mkSizeStmts stype fields ni = do -> fieldsInfo <- mapM getFieldInfo fields -> let anonSType = mkAnonFromDecl stype -> sizeBlocks <- liftM concat $ sequence $ map (uncurry3 $ mkSizeBlock ni) fieldsInfo -> sizeDecls <- mkDeclsFromRemote "AER_MK_SIZE_DECLS" [show $ pretty anonSType] ni -> startStmts <- mkStmtFromRemote "AER_MK_SIZE_STMTS_START" [show $ pretty anonSType] ni -> endStmts <- mkStmtFromRemote "AER_MK_SIZE_STMTS_END" [] ni -> return $ mkCompoundWithDecls Nothing sizeDecls (startStmts ++ sizeBlocks ++ endStmts) ni - -> mkSizeFun :: CDecl -> String -> [CDecl] -> NodeInfo -> RemoteT CExtDecl -> mkSizeFun decl sname fields ni = do -> let tparam = mkRemoteDecl decl [(CPtrDeclr [] ni)] "x" -> when (isNothing tparam) $ invalid "not a known encoding type" ni -> stmts <- mkSizeStmts (fromJust tparam) fields ni -> return $ mkFunDef ((CTypeDef (newIdent "uint64_t" ni) ni), []) -> --[(CStorageSpec (CStatic ni)), (CTypeQual (CInlineQual ni))] -> [] -> ("aer_encode_size_" ++ sname) -> [mkConstCDecl (CCharType ni) [CPtrDeclr [] ni] "varname" ni, -> mkCDecl (CVoidType ni) [CPtrDeclr [] ni] "vx" ni] -> stmts - -> mkPtrInitNullBlock :: NodeInfo -> String -> String -> String -> Bool -> RemoteT [CStat] -> mkPtrInitNullBlock ni _ typeName fieldName _ = do -> mkStmtFromRemote "AER_MK_INIT_PTR_NULL" [typeName, fieldName] ni - -> mkInitNullBlock :: NodeInfo -> String -> String -> String -> Bool -> RemoteT [CStat] -> mkInitNullBlock ni baseTypeName typeName fieldName isPtr = do -> let ptrParam = if isPtr then "" else "&" -> mkStmtFromRemote "AER_MK_INIT_NULL_TYPE" [baseTypeName, typeName, fieldName, ptrParam] ni - -> mkInitNullStmts :: CDecl -> [CDecl] -> NodeInfo -> RemoteT CStat -> mkInitNullStmts stype fields ni = do -> fieldsInfo <- mapM getFieldInfo fields -> let getDeriveds (CDecl _ declrs _) = concatMap getDerivedDeclrs declrs -> ptrFields = filter (\d -> any isDerivedPtr (getDeriveds d)) fields -> nonPtrFields = filter (\d -> not $ any isDerivedPtr (getDeriveds d)) fields -> anonSType = mkAnonFromDecl stype -> baseTypeName = getRemoteTypeName $ getTypeSpecFromDecl anonSType -> ptrFieldInfo <- mapM getFieldInfo ptrFields -> nonPtrFieldsInfo <- mapM getFieldInfo nonPtrFields -> when (isNothing baseTypeName) $ invalid ("type '" ++ (show $ pretty stype) ++ "does not have a known encoding type") ni -> nullBlocks <- liftM concat $ sequence $ map (uncurry3 $ mkPtrInitNullBlock ni $ fromJust baseTypeName) ptrFieldInfo -> initBlocks <- liftM concat $ sequence $ map (uncurry3 $ mkInitNullBlock ni $ fromJust baseTypeName) nonPtrFieldsInfo -> initDecls <- mkDeclsFromRemote "AER_MK_INIT_DECLS" [show $ pretty anonSType] ni -> startStmts <- mkStmtFromRemote "AER_MK_INIT_STMTS_START" [show $ pretty anonSType] ni -> endStmts <- mkStmtFromRemote "AER_MK_INIT_STMTS_END" [] ni - -> return $ mkCompoundWithDecls Nothing initDecls (startStmts ++ nullBlocks ++ initBlocks ++ endStmts) ni - -> mkInitNullFun :: CDecl -> String -> [CDecl] -> NodeInfo -> RemoteT CExtDecl -> mkInitNullFun decl sname fields ni = do -> fieldsInfo <- mapM getFieldInfo fields -> let tparam = mkRemoteDecl decl [(CPtrDeclr [] ni)] "x" -> when (isNothing tparam) $ invalid "not a known encoding type" ni -> stmts <- mkInitNullStmts (fromJust tparam) fields ni -> return $ mkFunDef ((CTypeDef (newIdent "triton_ret_t" ni) ni), []) -> --[(CStorageSpec (CStatic ni)), (CTypeQual (CInlineQual ni))] -> [] -> ("aer_init_null_" ++ sname) -> ([mkCDecl (CVoidType ni) [CPtrDeclr [] ni] "vx" ni]) -> stmts - -> mkPtrInitBlock :: NodeInfo -> String -> String -> String -> Bool -> RemoteT [CStat] -> mkPtrInitBlock ni baseTypeName typeName fieldName isPtr = do -> let ptrParam = if isPtr then "" else "&" -> mkStmtFromRemote "AER_MK_PTR_INIT_TYPE" [baseTypeName, typeName, fieldName, ptrParam] ni - -> mkInitBlock :: NodeInfo -> String -> String -> String -> Bool -> RemoteT [CStat] -> mkInitBlock ni baseTypeName typeName fieldName isPtr = do -> let ptrParam = if isPtr then "" else "&" -> mkStmtFromRemote "AER_MK_INIT_TYPE" [baseTypeName, typeName, fieldName, ptrParam] ni - -> mkInitStmts :: CDecl -> [CDecl] -> NodeInfo -> RemoteT CStat -> mkInitStmts stype fields ni = do -> fieldsInfo <- mapM getFieldInfo fields -> let getDeriveds (CDecl _ declrs _) = concatMap getDerivedDeclrs declrs -> anonSType = mkAnonFromDecl stype -> baseTypeName = getRemoteTypeName $ getTypeSpecFromDecl anonSType -> ptrFields = filter (\d -> any isDerivedPtr (getDeriveds d)) fields -> nonPtrFields = filter (\d -> not $ any isDerivedPtr (getDeriveds d)) fields -> ptrFieldInfo <- mapM getFieldInfo ptrFields -> nonPtrFieldsInfo <- mapM getFieldInfo nonPtrFields -> when (isNothing baseTypeName) $ invalid ("type '" ++ (show $ pretty stype) ++ "does not have a known encoding type") ni -> nullBlocks <- liftM concat $ sequence $ map (uncurry3 $ mkPtrInitBlock ni $ fromJust baseTypeName) ptrFieldInfo -> initBlocks <- liftM concat $ sequence $ map (uncurry3 $ mkInitBlock ni $ fromJust baseTypeName) nonPtrFieldsInfo -> initDecls <- mkDeclsFromRemote "AER_MK_INIT_DECLS" [show $ pretty anonSType] ni -> startStmts <- mkStmtFromRemote "AER_MK_INIT_STMTS_START" [show $ pretty anonSType] ni -> endStmts <- mkStmtFromRemote "AER_MK_INIT_STMTS_END" [] ni - -> return $ mkCompoundWithDecls Nothing initDecls (startStmts ++ nullBlocks ++ initBlocks ++ endStmts) ni - -> mkInitFun :: CDecl -> String -> [CDecl] -> NodeInfo -> RemoteT CExtDecl -> mkInitFun decl sname fields ni = do -> fieldsInfo <- mapM getFieldInfo fields -> let tparam = mkRemoteDecl decl [(CPtrDeclr [] ni)] "x" -> getDeriveds (CDecl _ declrs _) = concatMap getDerivedDeclrs declrs -> params = map (\d -> if (any isDerivedPtr (getDeriveds d)) then d else addPtrDeclr d) fields -> when (isNothing tparam) $ invalid "not a known encoding type" ni -> stmts <- mkInitStmts (fromJust tparam) fields ni -> return $ mkFunDef ((CTypeDef (newIdent "triton_ret_t" ni) ni), []) -> [] -> ("aer_init_" ++ sname) -> ([mkCDecl (CVoidType ni) [CPtrDeclr [] ni] "vx" ni] ++ params) -> stmts - -> mkPtrCopyBlock :: NodeInfo -> String -> String -> String -> Bool -> RemoteT [CStat] -> mkPtrCopyBlock ni baseTypeName typeName fieldName isPtr = do -> let ptrParam = if isPtr then "" else "&" -> mkStmtFromRemote "AER_MK_PTR_COPY_TYPE" [baseTypeName, typeName, fieldName, ptrParam] ni - -> mkCopyBlock :: NodeInfo -> String -> String -> String -> Bool -> RemoteT [CStat] -> mkCopyBlock ni baseTypeName typeName fieldName isPtr = do -> let ptrParam = if isPtr then "" else "&" -> mkStmtFromRemote "AER_MK_COPY_TYPE" [baseTypeName, typeName, fieldName, ptrParam] ni - -> mkCopyStmts :: CDecl -> [CDecl] -> NodeInfo -> RemoteT CStat -> mkCopyStmts stype fields ni = do -> fieldsInfo <- mapM getFieldInfo fields -> let getDeriveds (CDecl _ declrs _) = concatMap getDerivedDeclrs declrs -> anonSType = mkAnonFromDecl stype -> baseTypeName = getRemoteTypeName $ getTypeSpecFromDecl anonSType -> ptrFields = filter (\d -> any isDerivedPtr (getDeriveds d)) fields -> nonPtrFields = filter (\d -> not $ any isDerivedPtr (getDeriveds d)) fields -> ptrFieldInfo <- mapM getFieldInfo ptrFields -> nonPtrFieldsInfo <- mapM getFieldInfo nonPtrFields -> when (isNothing baseTypeName) $ invalid ("type '" ++ (show $ pretty stype) ++ "does not have a known encoding type") ni -> ptrInitBlocks <- liftM concat $ sequence $ map (uncurry3 $ mkPtrCopyBlock ni $ fromJust baseTypeName) ptrFieldInfo -> initBlocks <- liftM concat $ sequence $ map (uncurry3 $ mkCopyBlock ni $ fromJust baseTypeName) nonPtrFieldsInfo -> initDecls <- mkDeclsFromRemote "AER_MK_COPY_DECLS" [show $ pretty anonSType] ni -> startStmts <- mkStmtFromRemote "AER_MK_COPY_STMTS_START" [show $ pretty anonSType] ni -> endStmts <- mkStmtFromRemote "AER_MK_COPY_STMTS_END" [] ni - -> return $ mkCompoundWithDecls Nothing initDecls (startStmts ++ ptrInitBlocks ++ initBlocks ++ endStmts) ni - -> mkCopyFun :: CDecl -> String -> [CDecl] -> NodeInfo -> RemoteT CExtDecl -> mkCopyFun decl sname fields ni = do -> fieldsInfo <- mapM getFieldInfo fields -> let tparam = mkRemoteDecl decl [(CPtrDeclr [] ni)] "x" -> mkVoidParam n = mkCDecl (CVoidType ni) [CPtrDeclr [] ni] n ni -> when (isNothing tparam) $ invalid "not a known encoding type" ni -> stmts <- mkCopyStmts (fromJust tparam) fields ni -> return $ mkFunDef ((CTypeDef (newIdent "triton_ret_t" ni) ni), []) -> --[(CStorageSpec (CStatic ni)), (CTypeQual (CInlineQual ni))] -> [] -> ("aer_copy_" ++ sname) -> [mkVoidParam "vx", mkVoidParam "vv"] -> stmts - -> mkDestroyBlock :: NodeInfo -> String -> String -> String -> Bool -> RemoteT [CStat] -> mkDestroyBlock ni baseTypeName typeName fieldName isPtr = do -> let ptrParam = if isPtr then "" else "&" -> mkStmtFromRemote "AER_MK_DESTROY_TYPE" [baseTypeName, typeName, fieldName, ptrParam] ni - -> mkDestroyStmts :: CDecl -> [CDecl] -> NodeInfo -> RemoteT CStat -> mkDestroyStmts stype fields ni = do -> fieldsInfo <- mapM getFieldInfo fields -> let anonSType = mkAnonFromDecl stype -> destroyBlocks <- liftM concat $ sequence $ map (uncurry3 $ mkDestroyBlock ni (show $ pretty anonSType)) fieldsInfo -> destroyDecls <- mkDeclsFromRemote "AER_MK_DESTROY_DECLS" [show $ pretty anonSType] ni -> startStmts <- mkStmtFromRemote "AER_MK_DESTROY_STMTS_START" [show $ pretty anonSType] ni -> endStmts <- mkStmtFromRemote "AER_MK_DESTROY_STMTS_END" [] ni - -> return $ mkCompoundWithDecls Nothing destroyDecls (startStmts ++ destroyBlocks ++ endStmts) ni - -> mkDestroyFun :: CDecl -> String -> [CDecl] -> NodeInfo -> RemoteT CExtDecl -> mkDestroyFun decl sname fields ni = do -> let tparam = mkRemoteDecl decl [(CPtrDeclr [] ni)] "x" -> when (isNothing tparam) $ invalid "not a known encoding type" ni -> stmts <- mkDestroyStmts (fromJust tparam) fields ni -> return $ mkFunDef ((CVoidType ni), []) -> --[(CStorageSpec (CStatic ni)), (CTypeQual (CInlineQual ni))] -> [] -> ("aer_destroy_" ++ sname) -> [mkCDecl (CVoidType ni) [CPtrDeclr [] ni] "vx" ni] -> stmts - -> mkEncodingStruct :: CDecl -> String -> NodeInfo -> RemoteT CExtDecl -> mkEncodingStruct decl sname ni = do -> let ffs = zip fields funs -> fields = ["encode", "decode", "encode_size", "init_null", "copy", "destroy"] -> funs = ["aer_encode_"++sname, "aer_decode_"++sname, "aer_encode_size_"++sname, "aer_init_null_"++sname, "aer_copy_"++sname, "aer_destroy_"++sname] -> return $ mkFunPtrsStruct "aer_encoder" ("aer_encoder_"++sname++" __attribute__ ((unused))") ffs True ni - -> tripleFST :: (a,b,c) -> a -> tripleFST (a,b,c) = a - -> registerStruct :: CDecl -> RemoteT [CExtDecl] -> registerStruct d@(CDecl specs inits ni) = (arp_trace "registerStruct") $ do -> let si = getStructInfo d -> (Just (structName, fields)) = si -> s = getRemoteTypeName (getTypeSpec specs) -> sname = (arp_trace ("registerStruct: "++ (show structName))) fromJust s -> allFields = join $ map splitDecls fields - -> -- check that struct has a name -> when (isNothing s) $ invalid ("cannot define a remote struct without a name\n") ni - -> -- check that fields all have encoding functions -> finfos <- mapM getFieldInfo fields -> let types = map tripleFST finfos -> bools <- mapM isFullEncoding types -> let nInfos = map nodeInfo fields -> tbns = zip4 types bools nInfos fields -> let checkRemoteFields (t, b, n, f) = when (not b) $ -> invalid ("struct '" ++ (show $ fromJust structName) ++ "' with parameter '" ++ -> (show $ getCDeclName $ f) ++ "' is not a valid remote type\n") n -> mapM_ checkRemoteFields tbns -> -> let (s:_) = filter isStructTypeSpec specs -> if (structSpecHasFields s) -> then do -> let newS = CDeclExt $ CDecl (filterOutRemoteSpec specs) inits ni -> enc <- (arp_trace "mkEncodeFun") mkEncodeFun d sname allFields ni -> size <- mkSizeFun d sname allFields ni -> dec <- mkDecodeFun d sname allFields ni -> initNull <- mkInitNullFun d sname allFields ni -> init <- mkInitFun d sname allFields ni -> dest <- mkDestroyFun d sname allFields ni -> copy <- mkCopyFun d sname allFields ni -> encStruct <- mkEncodingStruct d sname ni -> decls <- mkStructFunDecls (CDeclExt d) -> registerFullEncoding sname ni -> addRemoteDecl newS ni -> return $ [newS] ++ decls ++ [dest, initNull, copy, size, enc, dec, init, encStruct] -> else return [] - -> transDecl :: CExtDecl -> RemoteT [CExtDecl] -> transDecl e@(CDeclExt d@(CDecl specs inits ni)) -> | any isRemoteSpec specs && isStructDecl d = (arp_trace "transDecl") registerStruct d -> | any isRemoteSpec specs = do -> invalid "__remote specifiers can only be placed on struct declarations.\n" ni -> return [] -> | otherwise = return [] - -> getFileName :: RemoteT String -> getFileName = do -> w <- get -> return $ filename w - -> setFileName :: FilePath -> RemoteT () -> setFileName f = do -> r <- get -> put $ r { filename = f } - -> getFilePosStr :: NodeInfo -> RemoteT String -> getFilePosStr (NodeInfo p _ _) = do -> fname <- getFileName -> return (fname ++ ":" ++ (show $ posRow p) ++ ":" ++ (show $ posColumn p)) - -> inRemoteStructs :: CDecl -> Bool -> inRemoteStructs _ = True - -> isRemoteFunDef :: CFunDef -> Bool -> isRemoteFunDef (CFunDef specs declr decls stmt ni) = any isRemoteSpec specs - -> filterOutRemoteSpec :: [CDeclSpec] -> [CDeclSpec] -> filterOutRemoteSpec = filter (not . isRemoteSpec) - -> filterOutTypeSpec :: [CDeclSpec] -> [CDeclSpec] -> filterOutTypeSpec = filter (not . isTypeSpec) - -> mkStubDecl :: String -> ReturnType -> [CDecl] -> NodeInfo -> RemoteT CExtDecl -> mkStubDecl fname r@(retType, retDerived) params ni = do -> [decl] <- liftM (map CDeclExt) $ mkDeclsFromRemote "AER_MK_STUB_DECL" ([(returnToString r), fname] ++ (map (show . pretty) params)) ni -> return decl - -> mkGSRetType :: NodeInfo -> CTypeSpec -> mkGSRetType ni = (CTypeDef (newIdent "triton_ret_t" ni) ni) - -> mkGSBufferType :: NodeInfo -> CTypeSpec -> mkGSBufferType ni = (CTypeDef (newIdent "triton_buffer_t" ni) ni) - -> checkRemoteParams :: NodeInfo -> [CDecl] -> RemoteT () -> checkRemoteParams ni [inputdecl, outputdecl] = do -> let inNI = nodeInfo inputdecl -> inName = identToString $ getCDeclName inputdecl -> outNI = nodeInfo outputdecl -> outName = identToString $ getCDeclName outputdecl -> when (not $ inRemoteStructs inputdecl) $ invalid ("input parameter '"++inName++"' cannot be encoded, must use a __remote type") inNI -> when (not $ inRemoteStructs outputdecl) $ invalid ("output parameter '"++outName++"' cannot be encoded, must use a __remote type") outNI -> let (CDecl specs initdeclrs ni) = outputdecl -> ptrs = filter isDerivedPtr $ join $ map getDerivedDeclrs initdeclrs -> when ((length ptrs) /= 1) $ invalid ("output parameter '"++outName++"' must be a pointer to type") outNI -> return () - -> checkRemoteParams ni _ = do -> invalid "only two parameters allowed for a __remote declaration" ni -> return () - -> isAEReturnType :: CTypeSpec -> Bool -> isAEReturnType (CTypeDef (Ident name _ _) _) = name == "triton_ret_t" - -> checkRemoteReturn :: (CTypeSpec, [CDerivedDeclr]) -> RemoteT () -> checkRemoteReturn (retType, retDerived) = do -> when (not $ isAEReturnType retType) $ invalid "__remote function must have a return type of triton_ret_t" (nodeInfo retType) -> return () - -> registerEFun :: String -> RemoteT () -> registerEFun fname = do -> let encRE = "aer_encode_(.*)$" -> decRE = "aer_decode_(.*)$" -> sizeRE = "aer_encode_size_(.*)$" -> initNullRE = "aer_init_null_(.*)$" -> copyRE = "aer_copy_(.*)$" -> destRE = "aer_destroy_(.*)$" -> getT r = fst ((head r) ! 1) - -> let encResult = (fname =~ encRE) :: [MatchText String] -> when (not $ null encResult) $ registerEncodeFun $ getT encResult - -> let decResult = (fname =~ decRE) :: [MatchText String] -> when (not $ null decResult) $ registerDecodeFun $ getT decResult - -> let sizeResult = (fname =~ sizeRE) :: [MatchText String] -> when (not $ null sizeResult) $ registerEncodeSizeFun $ getT sizeResult - -> let initNullResult = (fname =~ initNullRE) :: [MatchText String] -> when (not $ null initNullResult) $ registerInitNullFun $ getT initNullResult - -> let copyResult = (fname =~ copyRE) :: [MatchText String] -> when (not $ null copyResult) $ registerCopyFun $ getT copyResult - -> let destResult = (fname =~ destRE) :: [MatchText String] -> when (not $ null destResult) $ registerDestroyFun $ getT destResult - -> registerRemoteDecl :: CExtDecl -> RemoteT [CExtDecl] -> registerRemoteDecl e@(CDeclExt d@(CDecl specs inits ni)) - -> | isStructDecl d && (any isRemoteSpec specs) = (arp_trace ("registerRemoteDecl: structDecl: " ++ (fileAndLine ni))) transDecl e -> | isFunDecl d && (any isRemoteSpec specs) = (arp_trace ("registerRemoteDecl: funDecl: " ++ (fileAndLine ni))) $ do -> let (fname, returnType, params) = splitFunDecl d -> newspecs = filterOutRemoteSpec specs -> ni = nodeInfo e - -> -- only two params allowed right now: input and pointer to output -> checkRemoteParams ni params - -> checkRemoteReturn returnType - -> let [inputdecl, outputdecl] = params -> origDecl = CDeclExt $ (CDecl newspecs inits ni) -> stubDecl <- mkStubDecl (identToString fname) returnType params ni -> serviceDecls <- mkServiceDecl ni (identToString fname) - -> return $ [origDecl, stubDecl] ++ serviceDecls - -> | isFunDecl d = do -> let (fname, _, _) = splitFunDecl d -> registerEFun (identToString fname) -> return [e] - -> registerRemoteDecl e@(CFDefExt fd) = do -> registerEFun $ getFunDefName fd -> return [e] - -> registerRemoteDecl e = return [e] - -> registerRemoteDecls :: CTranslUnit -> RemoteT CTranslUnit -> registerRemoteDecls (CTranslUnit decls ni) = do -> newdecls <- liftM concat $ sequence $ map registerRemoteDecl decls -> return $ CTranslUnit newdecls ni - -> data ParserOpts = Pretty | Help | Report String | Outfile String | Infile String | ServiceName String | RegistryDir String | Compiler String - -> getCompiler :: [ParserOpts] -> String -> getCompiler ((Compiler s):ps) = s -> getCompiler (_:ps) = getCompiler ps -> getCompiler [] = "gcc" - -> getReportFilename :: [ParserOpts] -> Maybe String -> getReportFilename ((Report s):ps) = Just s -> getReportFilename (_:ps) = getReportFilename ps -> getReportFilename [] = Nothing - -> getOutfile :: [ParserOpts] -> Maybe String -> getOutfile ((Outfile s):ps) = Just s -> getOutfile (_:ps) = getOutfile ps -> getOutfile [] = Nothing - -> getInfiles :: [ParserOpts] -> [String] -> getInfiles ((Infile s):ps) = s:(getInfiles ps) -> getInfiles (_:ps) = getInfiles ps -> getInfiles [] = [] - -> getServiceName :: [ParserOpts] -> Maybe String -> getServiceName ((ServiceName s):ps) = Just s -> getServiceName (_:ps) = getServiceName ps -> getServiceName [] = Nothing - -> getRegistryDir :: [ParserOpts] -> Maybe String -> getRegistryDir ((RegistryDir s):ps) = Just s -> getRegistryDir (_:ps) = getRegistryDir ps -> getRegistryDir [] = Nothing - -> parserOpts :: [OptDescr ParserOpts] -> parserOpts = -> [ Option ['p'] ["pretty"] (NoArg Pretty) -> "output in pretty form without source line macros" -> , Option ['h','?'] ["help"] (NoArg Help) -> "help text" -> , Option ['c'] ["compiler"] (ReqArg (\s -> Compiler s) "<compiler path>") -> "filename to use when reporting errors" -> , Option ['r'] ["report"] (ReqArg (\s -> Report s) "<report filename>") -> "filename to use when reporting errors" -> , Option ['o'] ["outfile"] (ReqArg (\s -> Outfile s) "<output file>") -> "filename to write translated C code" -> , Option ['i'] ["infile"] (ReqArg (\s -> Infile s) "<input file>") -> "input file" -> , Option ['n'] ["name"] (ReqArg (\s -> ServiceName s) "<service name>") -> "service name to use for remote functions" -> , Option ['d'] ["regdir"] (ReqArg (\s -> RegistryDir s) "<registry dir>") -> "registry directory path to use for generated service files" -> ] - -> optPretty :: ParserOpts -> Bool -> optPretty (Pretty) = True -> optPretty _ = False - -> optHelp :: ParserOpts -> Bool -> optHelp (Help) = True -> optHelp _ = False - -> mkStubBlock :: String -> String -> String -> Bool -> String -> String -> NodeInfo -> RemoteT [CStat] -> mkStubBlock fname inTypeName inName isInPtr outTypeName outName ni = do -> let ptrParam = if isInPtr then "" else "&" -> mkStmtFromRemote "AER_MK_STUB_BLOCK" [fname, inTypeName, inName, outTypeName, outName, ptrParam] ni - -> mkStubStmts :: String -> [CDecl] -> NodeInfo -> RemoteT CStat -> mkStubStmts fname params ni = do -> let [inparam, outparam] = params -> (Ident inParamName _ _) = getCDeclName inparam -> (Ident outParamName _ _) = getCDeclName outparam -> inTypeName = canonCDecl (removeAPtr inparam) -> outTypeName = canonCDecl (removeAPtr outparam) -> (CDecl _ inDerived _) = inparam -> isInPtr = any isDerivedPtr (join $ map getDerivedDeclrs inDerived) -> stubBlock <- mkStubBlock fname inTypeName inParamName isInPtr outTypeName outParamName ni -> stubDecls <- mkDeclsFromRemote "AER_MK_STUB_DECLS" [fname] ni -> return $ mkCompoundWithDecls Nothing -> -- declarations -> stubDecls -> stubBlock -> ni - -> mkStubFun :: CFunDef -> RemoteT CExtDecl -> mkStubFun fundef = do -> let fname = getFunDefName fundef -> params = getFunDefParams fundef -> ni = nodeInfo fundef -> noremote = filterOutRemoteSpec $ getDeclSpecs fundef -> newspecs = filterOutTypeSpec $ noremote -> stubStmts <- mkStubStmts fname params ni -> return $ mkFunDef ((CTypeDef (newIdent "triton_ret_t" ni) ni), []) -- return type -> newspecs -- get the storage specs for the function -> ("remote_" ++ fname) -- the function name for the stub -> ((genCDecl "aer_remote_ctx_t" "ctx" ni) : -> ((genCDecl "triton_addr_t" "id" ni) : params)) -- parameters -> stubStmts - -> mkServiceFun :: CFunDef -> RemoteT [CExtDecl] -> mkServiceFun fundef = (arp_trace ("mkServiceFun: " ++ (getFunDefName fundef))) $ do -> let fname = getFunDefName fundef -> [inparam, outparam]= getFunDefParams fundef -> ni = nodeInfo fundef -> noremote = filterOutRemoteSpec $ getDeclSpecs fundef -> newspecs = filterOutTypeSpec $ noremote -> (Ident inParamName _ _) = getCDeclName inparam -> (Ident outParamName _ _) = getCDeclName outparam -> inTypeName = show $ pretty (removeAPtr $ emptyDeclrList inparam) -> outTypeName = show $ pretty (removeAPtr $ emptyDeclrList outparam) -> canonInType = getRemoteTypeName $ getTypeSpecFromDecl inparam -> canonOutType = getRemoteTypeName $ getTypeSpecFromDecl outparam -> (CDecl _ inDerived _) = inparam -> isInPtr = any isDerivedPtr (join $ map getDerivedDeclrs inDerived) -> ptrParam = if isInPtr then "" else "*" -> when (not $ isJust canonInType) $ -> invalid ("'" ++ inTypeName ++ "'" ++ -> " is not a recognized encoding type for input parameter '" ++ inParamName ++ "' to function: " ++ fname) ni -> when (not $ isJust canonOutType) $ -> invalid ("'" ++ outTypeName ++ "'" ++ -> " is not a recognized encoding type for output parameter '" ++ outParamName ++ "' to function: " ++ fname) ni -> [fdef] <- mkFunDefFromRemote (fname ++ "_service_block") [] "AER_MK_SERVICE_FNDEF" -> [fname, -> inTypeName, -> fromJust canonInType, -> inParamName, -> outTypeName, -> fromJust canonOutType, -> outParamName, -> ptrParam] ni -> [fdefInit] <- mkFunDefFromRemote (fname ++ "_init_block") [] "AER_MK_SERVICE_INIT" -> [fname, -> inTypeName, -> fromJust canonInType, -> inParamName, -> outTypeName, -> fromJust canonOutType, -> outParamName, -> ptrParam] ni -> [fdefDestroy] <- mkFunDefFromRemote (fname ++ "_destroy_block") [] "AER_MK_SERVICE_DESTROY" -> [fname, -> inTypeName, -> fromJust canonInType, -> inParamName, -> outTypeName, -> fromJust canonOutType, -> outParamName, -> ptrParam] ni -> return $ [fdefInit, fdefDestroy, addDeclSpecs fdef newspecs] - -> mkServiceCtx :: CFunDef -> RemoteT CExtDecl -> mkServiceCtx fundef = do -> let fname = getFunDefName fundef -> ni = nodeInfo fundef -> [structCtx] <- mkDeclsFromRemote "AER_MK_SERVICE_CTX" [fname] ni -> return $ CDeclExt structCtx - -> mkOpStmts :: String -> NodeInfo -> RemoteT [CStat] -> mkOpStmts fname ni = mkStmtFromRemote "AER_MK_OP_STMTS" [fname] ni - -> mkOpFun :: CFunDef -> RemoteT CExtDecl -> mkOpFun fundef = do -> let fname = getFunDefName fundef -> ni = nodeInfo fundef -> opfunStmts <- mkOpStmts fname ni -> return $ mkFunDef ((CTypeDef (newIdent "uint64_t" ni) ni), []) -> [] -> ("__get_op_id_" ++ fname) -> [] -> (mkCompoundStmt Nothing (opfunStmts) ni) - -> mkRegCtorFun :: String -> NodeInfo -> RemoteT CExtDecl -> mkRegCtorFun serviceName ni = do -> [fdef] <- mkFunDefFromRemote "" [] "AER_MK_REG_CTOR_FNDEF" [serviceName] ni -> return fdef - -> transformRemote :: CExtDecl -> RemoteT [CExtDecl] -> transformRemote e@(CFDefExt funDef) -> | isRemoteFunDef funDef = (arp_trace ("transformRemote: " ++ (getFunDefName funDef))) $ do -> let re = removeRemoteSpecFromFunDef e -> ni = nodeInfo funDef -> stub <- mkStubFun funDef -> servF <- mkServiceFun funDef -> servDecls <- mkServiceDecl ni $ getFunDefName funDef -> servCtx <- mkServiceCtx funDef -> let rstub = removeRemoteSpecFromFunDef stub -> rservF = map removeRemoteSpecFromFunDef servF -> opfundecl = mkOpIdDecl (nodeInfo funDef) (getFunDefName funDef) -> opfun <- mkOpFun funDef -> sop <- liftM (map CDeclExt) $ mkDeclsFromRemote "AER_MK_STATIC_OP_DECL" [getFunDefName funDef] ni -> registerRemoteFun (getFunDefName funDef) (getFunDefReturnMaybe funDef) (getFunDefParams funDef) -> return $ sop ++ [opfundecl, opfun, re, rstub] ++ servDecls ++ rservF ++ [servCtx] -> | otherwise = return [e] -> transformRemote e = return [e] - -> transformR :: CTranslUnit -> RemoteT CTranslUnit -> transformR (CTranslUnit decls ni) = (arp_trace "transformR") $ do -> newdecls <- liftM concat $ sequence $ map transformRemote decls -> return $ CTranslUnit newdecls ni - -> mkRegBlock :: NodeInfo -> String -> RemoteT [CStat] -> mkRegBlock ni s = do -> mkStmtFromRemote "AER_MK_REG_BLOCK" [s] ni - -> mkRegFun :: String -> NodeInfo -> RemoteT CExtDecl -> mkRegFun sname ni = do -> decls <- mkDeclsFromRemote "AER_MK_REG_DECLS" [sname] ni -> rs <- getRemoteNames -> stmts <- liftM concat $ sequence $ map (mkRegBlock ni) rs -> startStmts <- mkStmtFromRemote "AER_MK_REG_START" [sname] ni -> endStmts <- mkStmtFromRemote "AER_MK_REG_END" [sname] ni -> return $ mkFunDef ((CTypeDef (newIdent "triton_ret_t" ni) ni), []) -> [] -> ("aer_remote_register_" ++ sname) -> [mkAnonCDecl (CVoidType ni) [] ni] -> (mkCompoundWithDecls Nothing -> decls -> (startStmts ++ stmts ++ endStmts) -> ni) - -> mkOpIdDecl :: NodeInfo -> String -> CExtDecl -> mkOpIdDecl ni s = -> CDeclExt $ mkFunDeclWithDeclSpecs -> ("__get_op_id_" ++ s) -> (CTypeDef (newIdent "uint64_t" ni) ni) -> [] -> [CStorageSpec $ CExtern ni] -> [mkAnonCDecl (CVoidType ni) [] ni] - -> mkOpIdDecls :: NodeInfo -> RemoteT [CExtDecl] -> mkOpIdDecls ni = do -> rs <- getRemoteNames -> return $ map (mkOpIdDecl ni) rs - -> mkServiceDecl :: NodeInfo -> String -> RemoteT [CExtDecl] -> mkServiceDecl ni s = do -> liftM (map CDeclExt) $ mkDeclsFromRemote "AER_MK_SERVICE_DECL" [s] ni - -> mkServiceDecls :: NodeInfo -> RemoteT [CExtDecl] -> mkServiceDecls ni = do -> rs <- getRemoteNames -> liftM concat $ sequence $ map (mkServiceDecl ni) rs - -> addRegisterFun :: String -> NodeInfo -> RemoteT CTranslUnit -> addRegisterFun sname ni = do -> opidDecls <- mkOpIdDecls ni -> sDecls <- mkServiceDecls ni -> regf <- mkRegFun sname ni -> ctor <- mkRegCtorFun sname ni -> return $ CTranslUnit (opidDecls ++ sDecls ++ [regf, ctor]) ni -> -- return $ CTranslUnit (opidDecls ++ sDecls ++ [regf]) ni - -> mkInitDeclParamStr :: NodeInfo -> String -> String -> Bool -> String -> mkInitDeclParamStr ni typeName fieldName isPtr = -> let ptrParam = if isPtr then " * " else "" -> in (typeName ++ " " ++ ptrParam ++ " " ++ fieldName) - -> mkStructFunDecls :: CExtDecl -> RemoteT [CExtDecl] -> mkStructFunDecls (CDeclExt d@(CDecl specs inits ni)) = do -> let si = getStructInfo d -> (Just (structName, fields)) = si -> s = getRemoteTypeName (getTypeSpec specs) -> sname = fromJust s -> getDeriveds (CDecl _ declrs _) = concatMap getDerivedDeclrs declrs -> params = map (\d -> if (any isDerivedPtr (getDeriveds d)) then d else addPtrDeclr d) fields - -> mDecls <- mkDeclsFromRemote "AER_MK_STRUCT_DECLS" (sname : (map (show . pretty) fields)) ni -> initDecl <- mkDeclsFromRemote "AER_MK_STRUCT_INIT_DECL" (sname : (map (show . pretty) params)) ni -> return $ map CDeclExt (mDecls ++ initDecl) - -> mkStubs :: NodeInfo -> RemoteT CTranslUnit -> mkStubs ni = do -> r <- get -> let rdecls = reverse $ decls r -> funDecls <- mapM mkStructFunDecls rdecls -> let allDecls = concat $ zipWith (\a b -> a:b) rdecls funDecls -> rs <- getRemotes -> stubs <- sequence $ map (\((,,) f r p) -> mkStubDecl f r p ni) rs -> return $ CTranslUnit (allDecls ++ stubs) ni - -> generateAST :: FilePath -> IO CTranslUnit -> generateAST input_file = do -> arp_debug "generateAST..." -> input_stream <- readInputStream input_file -> arp_debug "generateAST done." -> let parse_result = parseC input_stream (position 0 input_file 1 1) -> case parse_result of -> Left parse_err -> error $ "Parse failed for input file: " ++ input_file ++ ": " ++ (show parse_err) -> Right ast -> return ast - -> getRegistryDecl :: String -> NodeInfo -> CTranslUnit -> getRegistryDecl s ni = CTranslUnit [ -> CDeclExt $ (mkFunDecl ("aer_remote_register_" ++ s) -> (CTypeDef (newIdent "triton_ret_t" ni) ni) [] -> [mkAnonCDecl (CVoidType ni) [] ni]) ] ni - -> writeRegistryFiles :: String -> String -> Maybe String -> RemoteT () -> writeRegistryFiles s sr r = do -> liftIO $ arp_debug "writeRegistryFiles:" -> let ni = mkNodeInfoOnlyPos $ initPos s -> let sourcefile = (if (isJust r) then ((fromJust r) ++ "/") else "") ++ sr ++ "_module.ae" -> headerfile = (if (isJust r) then ((fromJust r) ++ "/") else "") ++ sr ++ "_module.h" -> stubfile = (if (isJust r) then ((fromJust r) ++ "/") else "") ++ sr ++ "_stubs.hae" - -> regCTU <- addRegisterFun s ni - -> liftIO $ (writeFile sourcefile) $ "\ -> \\n\n\ -> \/* This is an auto-generated source file created by the ae-remote-parser tool. DO NOT MODIFY! */\n\n\ -> \#include \"" ++ headerfile ++ "\"\n\n" -> liftIO $ ((appendFile sourcefile) . show . pretty) regCTU -> liftIO $ appendFile sourcefile "\n\n" - -> liftIO $ (writeFile headerfile) $ "\ -> \\n\n\ -> \/* This is an auto-generated header created by the ae-remote-parser tool. DO NOT MODIFY! */\n\n\ -> \#ifndef ___" ++ s ++ "_h__ \n\ -> \#define ___" ++ s ++ "_h__ \n\ -> \#include <aesop/aesop.h> \n\ -> \#include \"src/remote/service.hae\" \n\ -> \\n\n" -> liftIO $ ((appendFile headerfile) . show . pretty) (getRegistryDecl s ni) -> liftIO $ appendFile headerfile $ " \n\ -> \\n\n\ -> \#endif \ -> \\n\n" - -> stubCTU <- mkStubs ni -> liftIO $ (writeFile stubfile) $ "\ -> \\n\n\ -> \/* This is an auto-generated header file create by the ae-remote-parser tool. DO NOT MODIFY! */\n\n\ -> \#ifndef ___" ++ s ++ "_stubs_hae__ \n\ -> \#define ___" ++ s ++ "_stubs_hae__ \n\ -> \#include <aesop/aesop.h> \n\ -> \#include \"src/remote/service.hae\" \n\ -> \\n\n" -> liftIO $ ((appendFile stubfile) . show . pretty) stubCTU -> liftIO $ appendFile stubfile $ " \n\ -> \\n\n\ -> \#endif \ -> \\n\n" - -parseRemote: p: Pretty? - outfile: Output file - report: Apparent filename - f: File to process - -> parseRemote :: Bool -> Maybe FilePath -> Maybe FilePath -> FilePath -> RemoteT () -> parseRemote p outfile report f = do -> liftIO $ arp_debug ("parseRemote: " ++ f) -> let r = if isJust report then fromJust report else f -> outf = fromJust outfile -> liftIO $ arp_debug ("parseRemote: outf: " ++ (show outf)) -> setFileName r -> ctu <- liftIO $ generateAST f -> ctuWithDecls <- registerRemoteDecls ctu -> liftIO $ arp_debug ("parseRemote: got Decls") -> transCTU <- transformR ctuWithDecls -> liftIO $ arp_debug ("parseRemote: got transCTU") -> when (isJust outfile) $ (arp_trace "parseRemote: outfile OK") $ do -> -- liftIO $ putStrLn ("parseRemote: showing: ") -> -- liftIO $ putStrLn (show (serialize transCTU)) -> -- liftIO $ putStrLn "SHOWN." -> liftIO $ arp_debug ("parseRemote: writing: " ++ outf ++ " " ++ (show p) ++ " " ++ outf) -> if p then liftIO $ ((writeFile outf) . show . pretty) transCTU -> else liftIO $ ((writeFile outf) . show . serialize) transCTU -> liftIO $ appendFile (fromJust outfile) "\n\n" -> liftIO $ arp_debug "parseRemote: write done." -> return () - -> main :: IO () -> main = do -> hSetBuffering stdout NoBuffering -> -- hPutStrLn stderr "REMOTE PARSER!" -> arp_debug "REMOTE PARSER!" -> args <- getArgs -> let (opts, gccopts, errs) = getOpt RequireOrder parserOpts args -> pretty = any optPretty opts -> help = any optHelp opts -> inputs = getInfiles opts -> outfile = getOutfile opts -> report = getReportFilename opts -> srvname = getServiceName opts -> regdir = getRegistryDir opts -> compiler = getCompiler opts -> header = "Usage: ae-remote-parser [OPTIONS...] files..." -> -> when (not $ null errs) $ ioError $ userError ((concat errs) ++ -> (usageInfo header parserOpts)) -> -> when help $ do { putStrLn $ usageInfo header parserOpts ; exitWith (ExitFailure 1) } -> when (null inputs) $ ioError $ userError "No input file specified." -> when (isNothing srvname) $ ioError $ userError "No srvname!" -> when (isNothing outfile && isNothing srvname) $ ioError $ userError "No output file specified." -> when (isJust outfile && length inputs > 1) $ ioError $ userError "Output file specified with more than one input file." - -> -- empty string here because the parseRemote function sets the filename for each file -> -- file compiler macroHeader opts -> r <- newRemoteState "" compiler "src/aesop/ae-remote-parser.h" gccopts -> arp_debug "newRemoteState done." -> r <- foldM (\rr i -> execStateT (parseRemote pretty outfile report i) rr) r inputs -> arp_debug "parseRemotes done." -> arp_debug ("creating modules for: " ++ (fromJust srvname)) -> when (isJust srvname) $ (arp_trace "runStateT") $ do { runStateT (writeRegistryFiles (replace '-' '_' (fromJust srvname)) (fromJust srvname) regdir) r; return () } -> --Justin assert (isJust $ remoteParser r) return () -> return () --Justin - - removeFile $ macheader $ fromJust $ remoteParser r - -vim: ts=8 sts=4 sw=4 expandtab diff --git a/code/src/aesop/parser/ae-test.lhs b/code/src/aesop/parser/ae-test.lhs deleted file mode 100644 index 68482a5..0000000 --- a/code/src/aesop/parser/ae-test.lhs +++ /dev/null @@ -1,38 +0,0 @@ -> module Main where -> import Language.C -> import Language.C.Data.Position -> import Language.C.System.GCC -- preprocessor used -> import Language.C.Data.Ident -> import System.Environment -> import System.Console.GetOpt -> import System.Exit -> import Data.Typeable -> import Data.Maybe -> import Data.Either -> import Data.HashTable -> import Data.Generics -> import Data.Generics.Schemes -> import qualified Data.ByteString -> import qualified Data.ByteString.Char8 -> import Control.Monad.State -> import Control.Exception -> import Data.List -> import CGen -> import CParse -> import CUtils -> import System.IO.Unsafe -> import Debug.Trace -> import Data.Tree -> import BlockingContext -> import Walker -> import ContextGen -> import Serialize -> import Header -> import Language.C.Data.Node -> import Language.C.Data.Position - -> main :: IO () -> main = do -> let decls = mkDeclsFromCPPMacro "/radix-homes/slang/dev/grayskull/code/src/gsl/" "common/triton-blocking.h" (mkNodeInfoOnlyPos (initPos "dummy")) "GSTESTDECLMACRO" ["blah"] -> writeFile "/tmp/out1" "" -> mapM_ ((appendFile "/tmp/out1") . show. pretty) (map CDeclExt decls) diff --git a/code/src/aesop/parser/module.mk.in b/code/src/aesop/parser/module.mk.in deleted file mode 100644 index a39484d..0000000 --- a/code/src/aesop/parser/module.mk.in +++ /dev/null @@ -1,14 +0,0 @@ -DIR := src/aesop/parser - -HSSRC += $(DIR)/CGen.lhs \ - $(DIR)/CParse.lhs \ - $(DIR)/CUtils.lhs \ - $(DIR)/BlockingContext.lhs \ - $(DIR)/Walker.lhs \ - $(DIR)/ContextGen.lhs \ - $(DIR)/Serialize.hs \ - $(DIR)/Header.lhs - -HSBINSRC += $(DIR)/ae-remote-parser.lhs - -MODNAME_$(DIR)/ae-remote-parser.lhs = RemoteParser diff --git a/code/src/aesop/parser/tests/blocking/parallelfor4.ae b/code/src/aesop/parser/tests/blocking/parallelfor4.ae deleted file mode 100644 index 87c2fbe..0000000 --- a/code/src/aesop/parser/tests/blocking/parallelfor4.ae +++ /dev/null @@ -1,93 +0,0 @@ -/* - * (C) 2009 The University of Chicago - * - * See COPYING in top-level directory. - */ - -#include <aesop/aesop.h> -#include "src/aesop/parser/tests/blocking/btest.hae" -#include <triton-list.h> -#include "src/common/resources/scheduling/sched.hae" - -struct e -{ - int v; - triton_list_link_t link; -}; - -static triton_sched_t sched; -static triton_mutex_t sched_mutex = TRITON_MUTEX_INITIALIZER; - -static __blocking triton_ret_t foo(void) -{ - triton_ret_t ret; - - triton_mutex_lock(&sched_mutex); - ret = triton_sched_wait(sched, &sched_mutex); - triton_mutex_unlock(&sched_mutex); - - return(ret); -} - -static __blocking void run_parfor( - void) -{ - triton_list_t list; - struct e example; - - triton_list_link_clear(&example.link); - example.v = 5; - - triton_list_init(&list); - triton_sched_init(&sched); - - triton_list_add_back(&example.link, &list); - - pwait - { - pprivate int i; - pprivate struct e *entry, *scratch; - pprivate triton_ret_t pret; - - triton_list_for_each_entry(entry, scratch, &list, struct e, link) - { - pbranch - { - printf("Running first pbranch.\n"); - while(1) - { - pret = foo(); - if(pret != TRITON_SUCCESS) - { - break; - } - } - } - } - pbranch - { - printf("Running last pbranch.\n"); - btest_sleep(1); - aesop_cancel_branches(); - } - } - - printf("pwait done\n"); -} - -__blocking int aesop_main(int argc, char **argv) -{ - run_parfor(); - - return 0; -} -btest_main_set(aesop_main, "btest", "sched", "timer"); - -/* - * Local variables: - * c-indent-level: 4 - * c-basic-offset: 4 - * End: - * - * vim: ft=c ts=8 sts=4 sw=4 expandtab - */ diff --git a/code/src/aesop/parser/tests/remote/module.mk.in b/code/src/aesop/parser/tests/remote/module.mk.in deleted file mode 100644 index 687884e..0000000 --- a/code/src/aesop/parser/tests/remote/module.mk.in +++ /dev/null @@ -1,16 +0,0 @@ -DIR_TESTREMOTE := src/aesop/parser/tests/remote - -ifneq (,$(BUILD_MPI)) - -AERTESTSRC += $(DIR_TESTREMOTE)/test-remote-fault-injector.aer \ - $(DIR_TESTREMOTE)/test-remote-service-stop.aer \ - $(DIR_TESTREMOTE)/test-remote-error-wrap.aer \ - $(DIR_TESTREMOTE)/test-remote-buffer-response.aer \ - $(DIR_TESTREMOTE)/test-remote-list.aer - -TEST_RESULTS += $(DIR_TESTREMOTE)/test-remote-list.result - -#AERTESTSRC += $(DIR_TESTREMOTE)/test-remote-mpi.aer - -endif # BUILD_MPI - diff --git a/code/src/aesop/parser/tests/remote/test-remote-buffer-response.aer b/code/src/aesop/parser/tests/remote/test-remote-buffer-response.aer deleted file mode 100644 index 51684e5..0000000 --- a/code/src/aesop/parser/tests/remote/test-remote-buffer-response.aer +++ /dev/null @@ -1,110 +0,0 @@ -#include <unistd.h> - -#include "src/common/triton-types.h" -#include <aesop/hints.h> -#include <aesop/aesop.h> -#include "src/remote/encoding.h" -#include "src/remote/remote.hae" -#include "src/net/triton-message.hae" -#include "src/net/triton-message-method.hae" -#include "src/common/resources/scheduling/sched.hae" -#include <aesop/timer.hae> -#include "src/zeroconf/zeroconf.h" -#include "src/net/mpi/mpi.hae" -#include "src/net/mpi/mpi-method.h" - -#include "src/remote/service.hae" -#include "src/common/triton-init.h" - - -/* normally you don't have to declare this, but we do here because - * it gets declared in <module>_module.h (myclientserver_module.h) - * which is generated by this source file. - */ -triton_ret_t aer_remote_register_test_remote_buffer_response(void); - -#include "mpi.h" - -__remote struct test_resp -{ - triton_buffer_t buffer; -}; - -__remote __blocking triton_ret_t buffer_maker(int32_t in, struct test_resp *resp); - -__remote __blocking triton_ret_t buffer_maker(int32_t in, struct test_resp *resp) -{ - triton_ret_t tret; - - /* just allocate an empty buffer to send in the response */ - tret = triton_buffer_alloc(&resp->buffer, 10); - - return(tret); -} - -__blocking void do_remote_test(aer_remote_ctx_t rctx) -{ - triton_ret_t ret; - int32_t result; - int rank; - triton_addr_t addr; - char addr_str[256]; - struct test_resp resp; - - MPI_Comm_rank(MPI_COMM_WORLD, &rank); - - if(rank == 0) - { - /* need a good way to wait for all the processes, so that - * we know the aer_service_requests calls have been started - * and will be able to receive messages. For now, we just sleep - * for a sec. - */ - ret = aesop_timer(1000); - triton_error_assert(ret); - - sprintf(addr_str, "mpi://1"); - - ret = triton_addr_lookup(addr_str, &addr); - triton_error_assert(ret); - - ret = remote_buffer_maker(rctx, addr, 0, &resp); - triton_error_assert(ret); - - if(ret == TRITON_SUCCESS) - { - aer_destroy_struct_test_resp(&resp); - } - triton_addr_free(addr); - } - else - { - /* service a single request */ - assert(rctx != NULL); - ret = aer_service_requests(rctx, 1); - } -} - -__blocking int aesop_main(int argc, char **argv) -{ - triton_ret_t ret; - ae_op_id_t op_id; - - ret = aer_remote_register_test_remote_buffer_response(); - triton_error_assert(ret); - - do_remote_test(AER_DEFAULT_CTX); - - return 0; -} -aesop_main_set(aesop_main); - - -/* - * Local variables: - * c-indent-level: 4 - * c-basic-offset: 4 - * End: - * - * vim: ts=8 sts=4 sw=4 expandtab - */ diff --git a/code/src/aesop/parser/tests/remote/test-remote-error-wrap.aer b/code/src/aesop/parser/tests/remote/test-remote-error-wrap.aer deleted file mode 100644 index ef32c88..0000000 --- a/code/src/aesop/parser/tests/remote/test-remote-error-wrap.aer +++ /dev/null @@ -1,121 +0,0 @@ -#include <unistd.h> - -#include "src/common/triton-types.h" -#include <aesop/hints.h> -#include <aesop/aesop.h> -#include "src/remote/encoding.h" -#include "src/remote/remote.hae" -#include "src/net/triton-message.hae" -#include "src/net/triton-message-method.hae" -#include "src/common/resources/scheduling/sched.hae" -#include <aesop/timer.hae> -#include "src/zeroconf/zeroconf.h" -#include "src/net/mpi/mpi.hae" -#include "src/net/mpi/mpi-method.h" - -#include "src/remote/service.hae" -#include "src/common/triton-init.h" - -/* test case: run with two mpi processes. One will act as a remote server, - * one will act as a remote client. A single rpc function will be executed - * that returns a triton_ret_t error code. The rpc function will use - * triton_error_wrap() to append additional context information to the - * error. - */ - -/* normally you don't have to declare this, but we do here because - * it gets declared in <module>_module.h (myclientserver_module.h) - * which is generated by this source file. - */ -triton_ret_t aer_remote_register_test_remote_error_wrap(void); - -#include "mpi.h" - -__remote __blocking triton_ret_t error_maker(triton_ignore_t* req, triton_ignore_t* resp); - -__remote __blocking triton_ret_t error_maker(triton_ignore_t* req, triton_ignore_t* resp) -{ - triton_ret_t tret; - - tret = TRITON_ERR_EXIST; - - return triton_error_wrap(tret, triton_node_null, "some extra information"); -} - -__blocking void do_remote_test(aer_remote_ctx_t rctx) -{ - triton_ret_t ret; - int rank, nprocs; - triton_addr_t addr; - char addr_str[256]; - - MPI_Comm_rank(MPI_COMM_WORLD, &rank); - MPI_Comm_size(MPI_COMM_WORLD, &nprocs); - - if(nprocs != 2) - { - fprintf(stderr, "Error: please run this test program with exactly 2 processes.\n"); - exit(1); - } - - if(rank == 0) - { - /* this is the client */ - - /* sleep briefly to wait for server to get ready */ - aesop_timer(1000); - - /* lookup address of server at rank 1 */ - sprintf(addr_str, "mpi://1"); - ret = triton_addr_lookup(addr_str, &addr); - triton_error_assert(ret); - - /* call remote function */ - ret = remote_error_maker(rctx, addr, NULL, NULL); - - if(triton_error_equal(TRITON_ERR_EXIST, ret)) - { - fprintf(stderr, "Error code is correct.\n"); - } - else - { - fprintf(stderr, "Error code is WRONG.\n"); - } - - /* print some information about the return code */ - triton_error_print(ret, "return code from RPC call"); - triton_addr_free(addr); - } - else - { - /* this is the server */ - - /* service a single request */ - assert(rctx != NULL); - ret = aer_service_requests(rctx, 1); - } -} - -__blocking int aesop_main(int argc, char **argv) -{ - triton_ret_t ret; - ae_op_id_t op_id; - - ret = aer_remote_register_test_remote_error_wrap(); - triton_error_assert(ret); - - do_remote_test(AER_DEFAULT_CTX); - - return 0; -} -aesop_main_set(aesop_main); - - -/* - * Local variables: - * c-indent-level: 4 - * c-basic-offset: 4 - * End: - * - * vim: ts=8 sts=4 sw=4 expandtab - */ diff --git a/code/src/aesop/parser/tests/remote/test-remote-fault-injector.aer b/code/src/aesop/parser/tests/remote/test-remote-fault-injector.aer deleted file mode 100644 index 2953c2e..0000000 --- a/code/src/aesop/parser/tests/remote/test-remote-fault-injector.aer +++ /dev/null @@ -1,231 +0,0 @@ -#include <unistd.h> - -#include "src/common/triton-types.h" -#include <aesop/hints.h> -#include <aesop/aesop.h> -#include "src/remote/encoding.h" -#include "src/remote/remote.hae" -#include "src/net/triton-message.hae" -#include "src/net/triton-message-method.hae" -#include "src/common/resources/scheduling/sched.hae" -#include <aesop/timer.hae> -#include "src/zeroconf/zeroconf.h" -#include "src/net/mpi/mpi.hae" -#include "src/net/mpi/mpi-method.h" -#include "src/net/fault-injector/fault-method.h" -#include "src/remote/service.hae" -#include "src/common/triton-init.h" -#include "src/remote/core_stubs.hae" -#include "src/remote/core_module.h" - -/* normally you don't have to declare this, but we do here because - * it gets declared in <module>_module.h (myclientserver_module.h) - * which is generated by this source file. - */ -triton_ret_t aer_remote_register_test_remote_fault_injector(void); - -#include "mpi.h" - -__remote __blocking triton_ret_t scatter_gather(int32_t in, int32_t *out); - -__remote __blocking triton_ret_t scatter_gather(int32_t in, int32_t *out) -{ - triton_ret_t ret, - ret1 = TRITON_SUCCESS, - ret2 = TRITON_SUCCESS; - triton_addr_t from, n1, n2; - triton_string_t fromstr; - int rank, r1, r2, size; - char n1str[100], n2str[100]; - int32_t res1 = 0, res2 = 0; - aer_remote_ctx_t rctx; - - ret = aesop_hints_get("triton.remote.context", sizeof(rctx), &rctx); - triton_error_assert(ret); - assert(rctx != NULL); - - MPI_Comm_rank(MPI_COMM_WORLD, &rank); - MPI_Comm_size(MPI_COMM_WORLD, &size); - - printf("scatter_gather: rank=%d, input=%d\n", rank, in); - - if(rank == 0) - { - r1 = 1; - r2 = 2; - } - else - { - r1 = rank * 2; - if(rank % 2 == 1) ++r1; - r2 = r1 + 2; - } - - sprintf(n1str, "fault://%d", r1); - ret = triton_addr_lookup(n1str, &n1); - triton_error_assert(ret); - - sprintf(n2str, "fault://%d", r2); - ret = triton_addr_lookup(n2str, &n2); - triton_error_assert(ret); - - pwait - { - if(r1 < size) - { - pbranch - { - printf("scatter_gather: rank=%d :: remote_scatter_gather(node=%d, in=%d)\n", rank, r1, in); - assert(rctx != NULL); - ret1 = remote_scatter_gather(rctx, n1, in, &res1); - if(ret1 != TRITON_SUCCESS) - { - pbreak; - } - printf("scatter_gather: rank=%d :: remote_scatter_gather(node=%d, in=%d) done!\n", rank, r1, in); - } - } - - if(r2 < size) - { - pbranch - { - printf("scatter_gather: rank=%d :: remote_scatter_gather(node=%d, in=%d)\n", rank, r2, in); - assert(rctx != NULL); - ret2 = remote_scatter_gather(rctx, n2, in, &res2); - if(ret2 != TRITON_SUCCESS) - { - pbreak; - } - printf("scatter_gather: rank=%d :: remote_scatter_gather(node=%d, in=%d) done!\n", rank, r2, in); - } - } - } - - triton_addr_free(n1); - triton_addr_free(n2); - if(ret1 == TRITON_SUCCESS && ret2 == TRITON_SUCCESS) - { - *out = res1 + res2 + rank + in; - printf("scatter_gather: rank=%d => out=%d\n", rank, *out); - ret = TRITON_SUCCESS; - } - else - { - ret = (ret1 == TRITON_SUCCESS) ? ret2 : ret1; - ret = triton_error_wrap(ret, triton_node_null, "Error at remote scatter_gather"); - } - - return ret; -} - -__blocking void do_remote_test(aer_remote_ctx_t rctx) -{ - triton_ret_t ret; - int32_t result; - int rank; - int i; - int size; - char addr_str[100]; - triton_addr_t addr; - triton_ignore_t in, out; - - MPI_Comm_rank(MPI_COMM_WORLD, &rank); - MPI_Comm_size(MPI_COMM_WORLD, &size); - - if(rank == 0) - { - /* need a good way to wait for all the processes, so that - * we know the aer_service_requests calls have been started - * and will be able to receive messages. For now, we just sleep - * for a sec. - */ - ret = aesop_timer(1000); - triton_error_assert(ret); - - /* must set the remote context as a hint because we're about to call - * scatter_gather as a local call - */ - ret = aesop_hints_put("triton.remote.context", sizeof(rctx), &rctx, 1); - triton_error_assert(ret); - - /* rank 0 calls local scatter_gather, which starts everything */ - ret = scatter_gather(50, &result); - triton_error_assert(ret); - printf("FINAL RESULT: %d\n", result); - /* shut down the other ranks which are acting as servers */ - printf("shutting down processes...\n"); - for(i=1; i<size; i++) - { - sprintf(addr_str, "fault://%d", i); - ret = triton_addr_lookup(addr_str, &addr); - triton_error_assert(ret); - - ret = remote_triton_core_server_shutdown(rctx, addr, in, &out); - triton_error_assert(ret); - triton_error_assert(ret); - triton_addr_free(addr); - } - printf("done shutting down.\n"); - } - else - { - /* service a single scatter/gather request */ - assert(rctx != NULL); - aer_service_engine_start(rctx); - } -} - -__blocking int aesop_main(int argc, char **argv) -{ - triton_ret_t ret; - ae_op_id_t op_id; - int scan_ret; - - if(argc != 5) - { - fprintf(stderr, - "Usage: test-remote-fault-injector <send timeout rate (%%)> <recv timeout rate (%%)> <send error rate (%%)> <recv error rate (%%)>\n"); - return(-1); - } - - triton_zeroconf_set("triton.net.fault_injector.base_method", "triton.net.mpi"); - - triton_zeroconf_set("triton.net.fault_injector.insert_send_timeouts_at_rate", argv[1]); - triton_zeroconf_set("triton.net.fault_injector.insert_recv_timeouts_at_rate", argv[2]); - triton_zeroconf_set("triton.net.fault_injector.insert_send_errors_at_rate", argv[3]); - triton_zeroconf_set("triton.net.fault_injector.insert_recv_errors_at_rate", argv[4]); - - /* triton_debug_stderr_enable("all"); */ - - ret = aer_remote_register_test_remote_fault_injector(); - triton_error_assert(ret); - ret = aer_remote_register_core(); - triton_error_assert(ret); - - do_remote_test(AER_DEFAULT_CTX); - - return 0; -} - -static triton_ret_t init_zeroconf_params(int argc, char** argv) -{ - triton_ret_t tret; - - tret = triton_zeroconf_set("aesop.remote.default_net", "fault"); - triton_error_assert(tret); - - return(TRITON_SUCCESS); -} - -aesop_main_set_with_init(init_zeroconf_params, aesop_main); - - -/* - * Local variables: - * c-indent-level: 4 - * c-basic-offset: 4 - * End: - * - * vim: ts=8 sts=4 sw=4 expandtab - */ diff --git a/code/src/aesop/parser/tests/remote/test-remote-list.aer b/code/src/aesop/parser/tests/remote/test-remote-list.aer deleted file mode 100644 index 86bf84a..0000000 --- a/code/src/aesop/parser/tests/remote/test-remote-list.aer +++ /dev/null @@ -1,262 +0,0 @@ - -#include <unistd.h> - -#include "src/common/triton-types.h" -#include <triton-list.h> -#include <aesop/hints.h> -#include <aesop/aesop.h> -#include "src/remote/encoding.h" -#include "src/remote/remote.hae" -#include "src/net/triton-message.hae" -#include "src/net/triton-message-method.hae" -#include "src/common/resources/scheduling/sched.hae" -#include <aesop/timer.hae> -#include "src/zeroconf/zeroconf.h" -#include "src/net/mpi/mpi.hae" -#include "src/net/mpi/mpi-method.h" -#include "src/remote/service.hae" -#include "src/common/triton-init.h" - - -/* test case: run with two mpi processes. One will act as a remote server, - * one will act as a remote client. A single rpc function will be executed - * that returns a triton_list_t in the response. - * - */ - -/* normally you don't have to declare this, but we do here because - * it gets declared in <module>_module.h (myclientserver_module.h) - * which is generated by this source file. - */ -triton_ret_t aer_remote_register_test_remote_list(void); - -#include "mpi.h" - -/* The type of each element in the example linked list. Must be a __remote - * function in order to get encoded properly. - */ -__remote struct item -{ - int32_t x; - triton_string_t key; - triton_list_link_t list_link; -}; - -triton_list_t test_list; -triton_mutex_t test_mutex = TRITON_MUTEX_INITIALIZER; - - -/* Type: test_remote_list_t - * - * Encoding functions for a specific typedef (test_remote_list_t) of the - * underlying triton_list_t type. - */ -static inline uint64_t aer_encode_size_test_remote_list_t( - const char *n __unused__, void *x) -{ - uint64_t size; - triton_list_t* tmp_list = *(triton_list_t**)x; - - aer_encode_size_triton_list_t(tmp_list, NULL, struct item, struct_item, list_link, &size); - - return size; -} -static inline triton_ret_t aer_encode_test_remote_list_t( - triton_buffer_t *buf, const char *n __unused__, void *x) -{ - assert(buf); - assert(x); - triton_list_t* tmp_list = *(triton_list_t**)x; - triton_ret_t tret; - - aer_encode_triton_list_t(buf, n, tmp_list, struct item, struct_item, - list_link, &tret); - - return(tret); -} -static inline triton_ret_t aer_decode_test_remote_list_t( - triton_buffer_t *buf, char **n __unused__, void *x) -{ - assert(buf); - assert(x); - triton_list_t** tmp_list = (triton_list_t**)x; - triton_ret_t tret; - - aer_decode_triton_list_t(buf, n, (*tmp_list), struct item, struct_item, - list_link, &tret); - return (tret); -} -static inline triton_ret_t aer_init_null_test_remote_list_t(void *x) -{ - assert(x); - triton_list_t* tmp_list = *(triton_list_t**)x; - triton_list_init(tmp_list); - return TRITON_SUCCESS; -} -static inline triton_ret_t aer_copy_test_remote_list_t(void *x, void *v) -{ - triton_ret_t tret = TRITON_SUCCESS; - assert(x); - assert(v); - triton_list_t** dest_list = (triton_list_t**)x; - triton_list_t** src_list = (triton_list_t**)v; - - aer_copy_triton_list_t((*dest_list), (*src_list), struct item, - struct_item, list_link, &tret); - - return tret; -} -static inline void aer_destroy_test_remote_list_t(void *x) -{ - assert(x); - triton_list_t** tmp_list = (triton_list_t**)x; - - aer_destroy_triton_list_t((*tmp_list), struct item, struct_item, list_link); - - return; -} - -static struct aer_encoder __attribute__ ((unused)) aer_encoder_test_remote_list_t = -{ - .encode = aer_encode_test_remote_list_t, - .decode = aer_decode_test_remote_list_t, - .encode_size = aer_encode_size_test_remote_list_t, - .init_null = aer_init_null_test_remote_list_t, - .copy = aer_copy_test_remote_list_t, - .destroy = aer_destroy_test_remote_list_t -}; - -__remote struct test_resp -{ - test_remote_list_t resp_list; -}; - -__remote __blocking triton_ret_t list_maker(triton_ignore_t* req, struct test_resp* resp); - - -__remote __blocking triton_ret_t list_maker(triton_ignore_t* req, struct test_resp* resp) -{ - test_remote_list_t tmp_list= &test_list; - - /* lock to insure that the list isn't modified while we make a copy to - * send in the response. - */ - triton_mutex_lock(&test_mutex); - aer_init_struct_test_resp(resp, &tmp_list); - triton_mutex_unlock(&test_mutex); - - return (TRITON_SUCCESS); -} - -__blocking void do_remote_test(aer_remote_ctx_t rctx) -{ - triton_ret_t ret; - int rank, nprocs; - triton_addr_t addr; - char addr_str[256]; - struct test_resp resp; - struct item* tmp_item; - struct item* scratch_item; - int i; - - MPI_Comm_rank(MPI_COMM_WORLD, &rank); - MPI_Comm_size(MPI_COMM_WORLD, &nprocs); - - if(nprocs != 2) - { - fprintf(stderr, "Error: please run this test program with exactly 2 processes.\n"); - exit(1); - } - - if(rank == 0) - { - /* this is the client */ - - /* sleep briefly to wait for server to get ready */ - aesop_timer(1000); - - /* lookup address of server at rank 1 */ - sprintf(addr_str, "mpi://1"); - ret = triton_addr_lookup(addr_str, &addr); - triton_error_assert(ret); - - /* call remote function */ - ret = remote_list_maker(rctx, addr, NULL, &resp); - - if(ret != TRITON_SUCCESS) - { - fprintf(stderr, "Error: remote function call failed.\n"); - } - else - { - triton_list_for_each_entry(tmp_item, scratch_item, resp.resp_list, - struct item, list_link) - { - printf("client got x value: %d\n", (int)tmp_item->x); - printf("client got key: %s\n", triton_string_get(&tmp_item->key)); - } - - /* NOTE: the client must make a copy of the data (or do whatever - * it wants with it) before calling the destroy function below. - */ - aer_destroy_struct_test_resp(&resp); - } - triton_addr_free(addr); - } - else - { - /* this is the server */ - - /* set up global list that we will try to transmit */ - triton_list_init(&test_list); - - for(i=0; i<3; i++) - { - tmp_item = malloc(sizeof(*tmp_item)); - assert(tmp_item); - tmp_item->x = i; - triton_string_init(&tmp_item->key, "String key: %d", i); - printf("server sending key: %s\n", triton_string_get(&tmp_item->key)); - triton_list_link_clear(&tmp_item->list_link); - triton_list_add_back(&tmp_item->list_link, &test_list); - } - - /* service a single request */ - assert(rctx != NULL); - ret = aer_service_requests(rctx, 1); - - /* tear down the data structure */ - triton_list_for_each_entry(tmp_item, scratch_item, &test_list, - struct item, list_link) - { - triton_list_del(&tmp_item->list_link); - triton_string_destroy(&tmp_item->key); - free(tmp_item); - } - triton_list_destroy(&test_list); - } -} - -__blocking int aesop_main(int argc, char **argv) -{ - triton_ret_t ret; - ae_op_id_t op_id; - - ret = aer_remote_register_test_remote_list(); - triton_error_assert(ret); - - do_remote_test(AER_DEFAULT_CTX); - - return 0; -} -aesop_main_set(aesop_main); - - -/* - * Local variables: - * c-indent-level: 4 - * c-basic-offset: 4 - * End: - * - * vim: ts=8 sts=4 sw=4 expandtab - */ diff --git a/code/src/aesop/parser/tests/remote/test-remote-list.sh b/code/src/aesop/parser/tests/remote/test-remote-list.sh deleted file mode 100755 index 07cfa5e..0000000 --- a/code/src/aesop/parser/tests/remote/test-remote-list.sh +++ /dev/null @@ -1,14 +0,0 @@ -#!/bin/bash - -[[ -n ${TRITON_SCRIPT_DEBUG} ]] && set -x - -shopt -s extglob -SCRIPT=$0 -BIN=${SCRIPT//+(${SRCDIR}\/|.sh)} -OUT=${BIN}.out - -mpiexec -l -n 2 ${VALGRIND} ${BIN} > ${OUT} 2>&1 -[[ $? == 0 ]] || exit 1 - -echo OK -exit 0 diff --git a/code/src/aesop/parser/tests/remote/test-remote-mpi.aer b/code/src/aesop/parser/tests/remote/test-remote-mpi.aer deleted file mode 100644 index 6d23a52..0000000 --- a/code/src/aesop/parser/tests/remote/test-remote-mpi.aer +++ /dev/null @@ -1,181 +0,0 @@ -#include <unistd.h> - -#include "src/common/triton-types.h" -#include <aesop/hints.h> -#include <aesop/aesop.h> -#include "src/remote/encoding.h" -#include "src/remote/remote.hae" -#include "src/net/triton-message.hae" -#include "src/net/triton-message-method.hae" -#include "src/common/resources/scheduling/sched.hae" -#include <aesop/timer.hae> -#include "src/zeroconf/zeroconf.h" -#include "src/net/mpi/mpi.hae" -#include "src/net/mpi/mpi-method.h" - -#include "src/remote/service.hae" -#include "src/common/triton-init.h" - - -/* normally you don't have to declare this, but we do here because - * it gets declared in <module>_module.h (myclientserver_module.h) - * which is generated by this source file. - */ -triton_ret_t aer_remote_register_test_remote_mpi(void); - -#include "mpi.h" - -__remote __blocking triton_ret_t scatter_gather(int32_t in, int32_t *out); - -__remote __blocking triton_ret_t scatter_gather(int32_t in, int32_t *out) -{ - triton_ret_t ret, - ret1 = TRITON_SUCCESS, - ret2 = TRITON_SUCCESS; - triton_addr_t from, n1, n2; - triton_string_t fromstr; - int rank, r1, r2, size; - char n1str[100], n2str[100]; - int32_t res1 = 0, res2 = 0; - aer_remote_ctx_t rctx; - - ret = aesop_hints_get("triton.remote.context", sizeof(rctx), &rctx); - triton_error_assert(ret); - assert(rctx != NULL); - - MPI_Comm_rank(MPI_COMM_WORLD, &rank); - MPI_Comm_size(MPI_COMM_WORLD, &size); - - printf("scatter_gather: rank=%d, input=%d\n", rank, in); - - if(rank == 0) - { - r1 = 1; - r2 = 2; - } - else - { - r1 = rank * 2; - if(rank % 2 == 1) ++r1; - r2 = r1 + 2; - } - - sprintf(n1str, "mpi://%d", r1); - ret = triton_addr_lookup(n1str, &n1); - triton_error_assert(ret); - - sprintf(n2str, "mpi://%d", r2); - ret = triton_addr_lookup(n2str, &n2); - triton_error_assert(ret); - - pwait - { - if(r1 < size) - { - pbranch - { - printf("scatter_gather: rank=%d :: remote_scatter_gather(node=%d, in=%d)\n", rank, r1, in); - assert(rctx != NULL); - ret1 = remote_scatter_gather(rctx, n1, in, &res1); - if(ret1 != TRITON_SUCCESS) - { - pbreak; - } - printf("scatter_gather: rank=%d :: remote_scatter_gather(node=%d, in=%d) done!\n", rank, r1, in); - } - } - - if(r2 < size) - { - pbranch - { - printf("scatter_gather: rank=%d :: remote_scatter_gather(node=%d, in=%d)\n", rank, r2, in); - assert(rctx != NULL); - ret2 = remote_scatter_gather(rctx, n2, in, &res2); - if(ret2 != TRITON_SUCCESS) - { - pbreak; - } - printf("scatter_gather: rank=%d :: remote_scatter_gather(node=%d, in=%d) done!\n", rank, r2, in); - } - } - } - - triton_addr_free(n1); - triton_addr_free(n2); - if(ret1 == TRITON_SUCCESS && ret2 == TRITON_SUCCESS) - { - *out = res1 + res2 + rank + in; - printf("scatter_gather: rank=%d => out=%d\n", rank, *out); - ret = TRITON_SUCCESS; - } - else - { - ret = (ret1 == TRITON_SUCCESS) ? ret2 : ret1; - ret = triton_error_wrap(ret, triton_node_null, "Error at remote scatter_gather"); - } - - return ret; -} - -__blocking void do_remote_test(aer_remote_ctx_t rctx) -{ - triton_ret_t ret; - int32_t result; - int rank; - - MPI_Comm_rank(MPI_COMM_WORLD, &rank); - - if(rank == 0) - { - /* need a good way to wait for all the processes, so that - * we know the aer_service_requests calls have been started - * and will be able to receive messages. For now, we just sleep - * for a sec. - */ - ret = aesop_timer(1000); - triton_error_assert(ret); - - /* must set the remote context as a hint because we're about to call - * scatter_gather as a local call - */ - ret = aesop_hints_put("triton.remote.context", sizeof(rctx), &rctx, 1); - triton_error_assert(ret); - - /* rank 0 calls local scatter_gather, which starts everything */ - ret = scatter_gather(50, &result); - triton_error_assert(ret); - printf("FINAL RESULT: %d\n", result); - - } - else - { - /* service a single scatter/gather request */ - assert(rctx != NULL); - ret = aer_service_requests(rctx, 1); - } -} - -__blocking int aesop_main(int argc, char **argv) -{ - triton_ret_t ret; - ae_op_id_t op_id; - - ret = aer_remote_register_test_remote_mpi(); - triton_error_assert(ret); - - do_remote_test(AER_DEFAULT_CTX); - - return 0; -} -aesop_main_set(aesop_main); - - -/* - * Local variables: - * c-indent-level: 4 - * c-basic-offset: 4 - * End: - * - * vim: ts=8 sts=4 sw=4 expandtab - */ diff --git a/code/src/aesop/parser/tests/remote/test-remote-service-stop.aer b/code/src/aesop/parser/tests/remote/test-remote-service-stop.aer deleted file mode 100644 index eabb1fb..0000000 --- a/code/src/aesop/parser/tests/remote/test-remote-service-stop.aer +++ /dev/null @@ -1,113 +0,0 @@ -#include <unistd.h> - -#include "src/common/triton-types.h" -#include <aesop/hints.h> -#include <aesop/aesop.h> -#include "src/remote/encoding.h" -#include "src/remote/remote.hae" -#include "src/net/triton-message.hae" -#include "src/net/triton-message-method.hae" -#include "src/common/resources/scheduling/sched.hae" -#include <aesop/timer.hae> -#include "src/zeroconf/zeroconf.h" -#include "src/net/mpi/mpi.hae" -#include "src/net/mpi/mpi-method.h" -#include "src/common/triton-init.h" - -#include "src/remote/service.hae" - -#include "mpi.h" - -__remote __blocking triton_ret_t foobar(int32_t in, int32_t *out); - -__remote __blocking triton_ret_t foobar(int32_t in, int32_t *out) -{ - aesop_timer(10); - *out = in + 1; - - return TRITON_SUCCESS; -} - - -__blocking void do_remote_test(void) -{ - triton_ret_t ret; - - pwait - { - pbranch - { - aer_service_engine_start(AER_DEFAULT_CTX); - } - pbranch - { - aesop_timer(5000); - aer_service_engine_stop(); - } - } - - return; -} - -static int done = 0; -static void cb(void *up) -{ - ae_context_t ctx = (ae_context_t)up; - done=1; - ae_poll_break(ctx); -} - -triton_ret_t aer_remote_register_test_remote_service_stop(void); - -int main(int argc, char *argv[]) -{ - triton_ret_t ret; - ae_op_id_t op_id; - ae_context_t mpi_resource_ctx; - ae_hints_t hints; - - ret = triton_init("triton.client"); - triton_error_assert(ret); - - if(argc > 1) - { - triton_debug_enable("/dev/stdout", "all"); - } - if(argc > 2) - { - sleep(30); - } - - ret = aer_service_init(); - triton_error_assert(ret); - - - ret = aer_remote_register_test_remote_service_stop(); - triton_error_assert(ret); - - ae_hints_init(&hints); - - ae_context_create(&mpi_resource_ctx, "mpi", "sched", "timer"); - ret = ae_post_blocking(do_remote_test, cb, mpi_resource_ctx, &hints, mpi_resource_ctx, &op_id); - if(ret == TRITON_SUCCESS) - { - while(!done) - { - ae_poll(mpi_resource_ctx, 0); - } - } - triton_error_assert(ret); - - ae_hints_destroy(&hints); - triton_finalize(); - return 0; -} - -/* - * Local variables: - * c-indent-level: 4 - * c-basic-offset: 4 - * End: - * - * vim: ts=8 sts=4 sw=4 expandtab - */ diff --git a/code/src/aesop/parser/tests/syntax/err-untyped-func.ae b/code/src/aesop/parser/tests/syntax/err-untyped-func.ae deleted file mode 100644 index c8cf290..0000000 --- a/code/src/aesop/parser/tests/syntax/err-untyped-func.ae +++ /dev/null @@ -1,18 +0,0 @@ - -/* - * Error case: User omits function type - */ - -#include <stdio.h> - -#include <aesop.h> - -__blocking /* NO TYPE! */ f(int i) -{ - printf("HI\n"); -} - -int main(void) -{ - return 0; -} diff --git a/code/src/aesop/parser/tests/syntax/err-untyped-func.errs b/code/src/aesop/parser/tests/syntax/err-untyped-func.errs deleted file mode 100644 index febf5a0..0000000 --- a/code/src/aesop/parser/tests/syntax/err-untyped-func.errs +++ /dev/null @@ -1 +0,0 @@ -function lacks return type diff --git a/code/src/aesop/parser/tests/syntax/run-tests.sh b/code/src/aesop/parser/tests/syntax/run-tests.sh deleted file mode 100755 index eb0d59c..0000000 --- a/code/src/aesop/parser/tests/syntax/run-tests.sh +++ /dev/null @@ -1,48 +0,0 @@ -#!/bin/bash - -# AESOP SYNTAX TEST SUITE - -# Allows for compile errors - -# usage: assert <CODE> <MESSAGE> -# If code is non-zero, print message and exit -assert() -{ - local CODE=$1 - shift - if (( ${CODE} )) - then - echo ${*} - exit ${CODE} - fi -} - -INCLUDES="-I . -I $PWD/src/aesop -I /home/wozniak/sfw/openpa-1.0.3/include" - -DIR=src/aesop/parser/tests/syntax - -TESTS=( - err-untyped-func -) - -for TEST in ${TESTS} -do - echo - echo "Compiling: ${TEST}" - AECC_OUT=${DIR}/${TEST}.aecc.out - aecc ${DIR}/${TEST}.ae -- ${INCLUDES} > ${AECC_OUT} 2>&1 - CODE=${?} - ERRFILE=${DIR}/${TEST}.errs - if [[ -f ${ERRFILE} ]] - then - assert $(( ! CODE )) "Test should not have compiled!" - ERRS=$( cat ${ERRFILE} ) - grep -q "${ERRS}" ${AECC_OUT} - CODE=${?} - assert ${CODE} "Compiler did not produce correct message!" - else - assert ${CODE} - fi - echo -done - diff --git a/code/src/rebuild/module.mk.in b/code/src/rebuild/module.mk.in index fb2321a..d6c8164 100644 --- a/code/src/rebuild/module.mk.in +++ b/code/src/rebuild/module.mk.in @@ -1,13 +1,9 @@ DIR_REBUILD := src/rebuild -ifeq ($(BUILD_MPI),1) +#AELIBSRC += $(DIR_REBUILD)/rebuild.ae -AELIBSRC += $(DIR_REBUILD)/rebuild.ae +#AERLIBSRC += $(DIR_REBUILD)/rb.aer +#MODCFLAGS_$(DIR_REBUILD)/rb = $(MPICFLAGS) +#MODCC_$(DIR_REBUILD)/rb = $(MPICC) -AERLIBSRC += $(DIR_REBUILD)/rb.aer - -MODCFLAGS_$(DIR_REBUILD)/rb = $(MPICFLAGS) -MODCC_$(DIR_REBUILD)/rb = $(MPICC) - -endif diff --git a/code/src/remote/module.mk.in b/code/src/remote/module.mk.in index 540d036..fa3e751 100644 --- a/code/src/remote/module.mk.in +++ b/code/src/remote/module.mk.in @@ -9,9 +9,10 @@ AESOP_HDR += $(DIR_REMOTE)/message.hae \ AELIBSRC += $(DIR_REMOTE)/message.ae \ $(DIR_REMOTE)/service.ae \ - $(DIR_REMOTE)/retry-method.ae \ - $(DIR_REMOTE)/client-server-exec.ae \ - $(DIR_REMOTE)/core-client.ae + $(DIR_REMOTE)/retry-method.ae + +# $(DIR_REMOTE)/core-client.ae +# $(DIR_REMOTE)/client-server-exec.ae \ ifneq (,$(USE_MERCURY)) AESOP_HDR += $(DIR_REMOTE)/mercury-engine.hae @@ -22,5 +23,5 @@ AETESTSRC += $(DIR_REMOTE)/mercury-svr.ae AETESTSRC += $(DIR_REMOTE)/mercury-client.ae endif -AERLIBSRC += $(DIR_REMOTE)/core.aer +# AERLIBSRC += $(DIR_REMOTE)/core.aer diff --git a/code/src/replicated-osd/module.mk.in b/code/src/replicated-osd/module.mk.in index 9366972..60b441f 100644 --- a/code/src/replicated-osd/module.mk.in +++ b/code/src/replicated-osd/module.mk.in @@ -1,18 +1,16 @@ DIR_ROSD := src/replicated-osd -AESOP_HDR += $(DIR_ROSD)/rosd-trans-nr-cache.hae \ - $(DIR_ROSD)/rosd-test-harness.hae +AESOP_HDR += $(DIR_ROSD)/rosd-trans-nr-cache.hae -AELIBSRC += $(DIR_ROSD)/rosd-trans-nr-cache.ae \ - $(DIR_ROSD)/rosd-test-harness.ae \ - $(DIR_ROSD)/rosd-client.ae +# $(DIR_ROSD)/rosd-test-harness.hae -ifneq (,$(BUILD_MPI)) -LIBSRC += $(DIR_ROSD)/rosd-utils.c +AELIBSRC += $(DIR_ROSD)/rosd-trans-nr-cache.ae -AERLIBSRC += $(DIR_ROSD)/rosd.aer +# $(DIR_ROSD)/rosd-client.ae +# $(DIR_ROSD)/rosd-test-harness.ae \ -MODCFLAGS_$(DIR_ROSD)/rosd = $(MPICFLAGS) -MODCC_$(DIR_ROSD)/rosd = $(MPICC) +LIBSRC += $(DIR_ROSD)/rosd-utils.c -endif # BUILD_MPI +#AERLIBSRC += $(DIR_ROSD)/rosd.aer +#MODCFLAGS_$(DIR_ROSD)/rosd = $(MPICFLAGS) +#MODCC_$(DIR_ROSD)/rosd = $(MPICC) diff --git a/code/src/state/module.mk.in b/code/src/state/module.mk.in index eebf758..8941d57 100644 --- a/code/src/state/module.mk.in +++ b/code/src/state/module.mk.in @@ -1,12 +1,9 @@ DIR_STATE := src/state -ifeq ($(BUILD_MPI),1) +# AELIBSRC += $(DIR_STATE)/status.ae -AELIBSRC += $(DIR_STATE)/status.ae +# AERLIBSRC += $(DIR_STATE)/sc.aer -AERLIBSRC += $(DIR_STATE)/sc.aer +# MODCFLAGS_$(DIR_STATE)/sc = $(MPICFLAGS) +# MODCC_$(DIR_STATE)/sc = $(MPICC) -MODCFLAGS_$(DIR_STATE)/sc = $(MPICFLAGS) -MODCC_$(DIR_STATE)/sc = $(MPICC) - -endif # BUILD_MPI hooks/post-receive --

1 0

branch, devel-mercury, updated. 6570093903f4ddbf16cf2f71c266b46b245dbb0b
by noreply＠mcs.anl.gov 31 Jul '13

31 Jul '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, devel-mercury has been updated via 6570093903f4ddbf16cf2f71c266b46b245dbb0b (commit) from 36f81ca73a49061d9835e4c97419f0685ea4b707 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 6570093903f4ddbf16cf2f71c266b46b245dbb0b Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jul 31 15:24:07 2013 -0400 test commit ----------------------------------------------------------------------- Summary of changes: code/INSTALL | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) Diff of changes: diff --git a/code/INSTALL b/code/INSTALL index e8ded7e..8f2a0ad 100644 --- a/code/INSTALL +++ b/code/INSTALL @@ -1,3 +1,4 @@ + External prerequisites ============================== hooks/post-receive --

1 0

branch, devel-mercury, created. 36f81ca73a49061d9835e4c97419f0685ea4b707
by noreply＠mcs.anl.gov 31 Jul '13

31 Jul '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, devel-mercury has been created at 36f81ca73a49061d9835e4c97419f0685ea4b707 (commit) - Log ----------------------------------------------------------------- commit 36f81ca73a49061d9835e4c97419f0685ea4b707 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jul 18 20:06:48 2013 -0500 destroy hints in post_blocking() callback not entirely sure if this is supposed to be safe but it seems to work... commit dd184808bb2caaf316630283c9d7a8d9ff3d177c Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jul 18 19:52:20 2013 -0500 fix hints usage with post_blocking commit b6f73a1601f544c7d7f4072390638ab825732d35 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jul 18 17:09:01 2013 -0500 temporary workaround for bug commit 915e2d40bce8147ce369f1c0c4020e86b0395141 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jul 18 17:01:49 2013 -0500 expand rpc example to fwd between svrs commit 7a266c8f4af3dbae134e9b3ddda76a90b2194c78 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jul 18 15:40:20 2013 -0500 some reorganization - move registration into a shared routine in rpc-example.ae so that it can be used by clients and server interchangably commit 01629a4543fb3b9a0b2a0d3035cd92f810e47b87 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jul 18 15:18:59 2013 -0500 specify addresses at runtime commit db2c28c1cf26c530989ac30b8a5f5a1221da6b24 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jul 18 15:04:58 2013 -0500 pass in listening address to engine init commit ac8242b89bda38c0a5febd5a7976148c0503846c Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jul 18 10:00:29 2013 -0500 tweak to match latest mercury api/headers commit 771b72a86ea86dc1256beda99ca6357d1df3663b Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jul 17 23:12:48 2013 -0500 remove unused #include commit aee2a17e4b8acdb97a837077ab2b772a710d10e5 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jul 17 20:45:02 2013 -0500 update tosd concurrent io benchmark commit f81967de609a8f04c95478bab593087086e208cf Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jul 17 12:56:26 2013 -0500 fix misc. warnings commit 88f8e071970c834885ac6efe61ba93970bc22446 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jul 17 12:28:40 2013 -0500 turn off explicit warning for now commit c0ba1666ca52dde5ed902d4a487f1b450fd4112e Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jul 17 12:28:29 2013 -0500 silence warning about byteswap macro commit 108e69946fab783d7ffda5fca10db24df3b0c5da Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jul 17 11:19:59 2013 -0500 remove old triton-init infrastructure commit b16aea77c8a7e9275a3aae116290b1fd2c5dcadb Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Jul 16 22:10:00 2013 -0500 convert addr_t to node_t in triton errors - addr_t is not portable across the network, it is a local identifier - this also breaks the awkward dependency upon networking code in the triton return codes commit c0fe5007de4d92b49debf6a238a977598494a0b0 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Jul 16 21:14:42 2013 -0500 remove unused variable commit be69d68bd019f798aace3e649a8604fd4c28246b Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Jul 16 21:14:10 2013 -0500 move node.[ch] to src/common commit dc144eb605eb4378c6c26866e7e83f7590846b44 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 27 23:07:43 2013 -0400 bug fix commit 0339f4d8dab93cc70db906886f5941e276646460 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 27 23:00:35 2013 -0400 more debugging prints commit 3a010f8e2dbf0f51c0550aea53b60f667451f74f Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 27 22:50:40 2013 -0400 fill in rest of client test doesn't work yet commit 0bc76916886c404d948fd4c1838b75c4c7320c8b Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 27 22:42:44 2013 -0400 incremental progress on client test commit 6097a924d94cb7eff63acc00804240d227d04296 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 27 22:11:52 2013 -0400 skeleton for mercury client test commit 5c669a2dca6f32c17c72ccba7f7bce1d4d7e28fe Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 27 22:04:55 2013 -0400 draft implementation of __blocking mercury wait untested commit aa4f6e103a466eadc1cf677e9ed0ad2493a30f89 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 27 16:36:13 2013 -0400 stub in thread for client engine commit 36f49c03afec3b4b68d58b75fd898a95d886eff2 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 27 15:57:03 2013 -0400 fix init order and switch back to signal shutdown commit fa71b47155dcf8ec1d8dd321b59803ffb63e6ad0 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 27 15:50:29 2013 -0400 reorg a little bit for client/server operation commit b9926afb604f51ec2ee694182bd982ab8f970a13 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 27 14:40:20 2013 -0400 timer instead of signal to keep server running commit 038e08d33c3265a72a41fe6b6581cc683526f092 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 27 14:29:43 2013 -0400 register rpc handler commit d14a6a4c27ca2bb43a748f8568316304bec166dd Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 27 14:22:34 2013 -0400 initialize timer commit 0c4a0fdcc3175e34bd09ec9aa755ef560e54f6cd Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 27 14:12:26 2013 -0400 example __blocking function as rpc handler untested; need to write a client commit ae953e8e6134952d578ed67290e7b05747bece69 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 27 12:02:57 2013 -0400 stubs for example rpc function code commit 63b1ddd6ba91e68c406d75c9fc6af0cbf662e4e4 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 27 11:42:04 2013 -0400 fix cut and paste error commit 7717ae2d6a917e704b92d4f3a43f7f28d000ffd2 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 27 11:36:40 2013 -0400 thread to drive mercury server-side engine commit 3950b983465668b3c74a23e14403fd89d56edde6 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 27 10:52:59 2013 -0400 use signal resource to gracefully stop mercury svr commit a36d1ac8b63efc715b6c3e71f839654bdaa44b03 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jun 26 22:07:16 2013 -0400 init and finalize mercury svr engine commit 11aa5f440a2b03b2fecea3d7888a069abd99836a Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jun 26 22:06:05 2013 -0400 hard code BMI paths temporarily commit 2de95fd20ecffd7f9385ec9eacf308c71ec5dd81 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jun 26 21:37:03 2013 -0400 stub server side mercury engine commit 78ba3f78e3ae74a6dfe3aff0b5f50b8cf72cb077 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jun 26 21:24:56 2013 -0400 --with-mercury option to configure commit 1d6834bbaec12013f0431b791dc5bca4e376cdd6 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 20 16:26:24 2013 -0400 draft summary installation instructions commit cac2ce49468986a08694ccfa03ac45819a6d109c Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jun 20 14:44:40 2013 -0400 use aesop pkg-config for libraries also remove --with-openpa configure argument; opa is not used directly by triton so it can be specified by aesop pkg-config if needed commit 09a35c6adce3ee9ebc4d01ae53e7939e75b4090b Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Jun 18 22:09:37 2013 -0400 adjust c-utils header paths commit 2b628f55151922d9dfd7a47be52ac4ac6431dccb Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Jun 18 21:44:19 2013 -0400 add --with-c-utils option to autoconf commit cf5f47a33038eecaea3f59140354161b20dd5da0 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Mar 6 22:26:26 2013 -0500 fix mutex bug in mpi-method commit 28cd8ccafe302be7ac8fa6465aa87e95edf97e33 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Mar 6 22:08:30 2013 -0500 destroy strings before re-initializing commit 06c6c97e17dc63fc94627c0cfb82a8e09e092efc Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Mar 6 13:05:00 2013 -0500 bug fix: correct error code on cancel commit 822ac59fde72f938102b90e4b22b657da6b8997e Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Mar 6 12:22:00 2013 -0500 fix memory error commit ec68ec03c4d8d102eaf516ff6eb11e8b667ab0ae Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Mar 6 11:04:30 2013 -0500 remove old mpi resource include commit 062dbc8ccf9540d4380d44f3e99843e830b8f3b3 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Mar 4 17:20:55 2013 -0500 bug fix to fault injector error handling commit ee616c1675ed014a9735d10faa5eb9bc5ed77b77 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Mar 4 17:18:23 2013 -0500 incremental progress towards fault testing commit 30ac119a0585f10c476626dcf40d058db9e90dd9 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Mar 4 16:45:02 2013 -0500 various updates to fault-method; untested commit c9dbfc9dd8bc8b31cd206759afdfa3e27f98b5eb Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Mar 4 15:26:39 2013 -0500 update phist commit 543ccdb5e30d9a88443f6de0de5331ab88fd1043 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Mar 4 15:20:56 2013 -0500 update phist-client commit bc7c64986cdece66da5d1271aee8e1d7749a9980 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Mar 4 15:11:20 2013 -0500 update phist-server.ae commit 61e39beb8d7d8903dc367d3005f4f24999915cc8 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Feb 28 15:59:34 2013 -0500 update to reflect removal of aesop contexts commit 1b9ca34a1ce327c8a4fdd02f76f7a2cc677be1fe Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Feb 26 16:21:05 2013 -0500 more safety against aesop callback finalizing MPI commit 64589bdf5efbb6c815517752e2819abdf6898a49 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Feb 26 16:07:01 2013 -0500 put worker thread in charge of freeing datatypes commit 5e7f5252a1000752995df1c0fdd76975a85912b6 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Feb 26 14:58:27 2013 -0500 use MPI_Finalize() to check for callback race commit f2ba3a347c9f6354e67f5a20b38068ae125b9a82 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Feb 26 14:14:19 2013 -0500 use aesop_clear_cancel() to wait for cancellation commit 729991f6b7bd139e794ea2498fa76a60160ed475 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Feb 25 16:50:01 2013 -0500 remove some old unused aesop stuff commit b42f1e3234dcf325d0be6d457bdaab78863d67e7 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Feb 25 16:36:33 2013 -0500 enable cancellation tests for signal resource commit 4fbd43013bf28dbeb817ba7be6ef5ee9d98530fa Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Feb 25 16:27:16 2013 -0500 check cancellation on signal post commit e15c902b1cb952f26404dc804cb173dda3a0d085 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Feb 25 16:21:37 2013 -0500 avoid use of opcache in signal resource commit 6d4affdc7228cc20c739818968d245356aba39b4 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Feb 25 16:06:14 2013 -0500 fix init dependencies in signal resource commit 8bbcf2a1c1deb536cb4e5921c636421808df045b Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Feb 25 15:55:57 2013 -0500 tweak init check in signal resource commit 43c2975cef24c0e60107485359eb0caa5aa3a09f Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Feb 25 15:50:26 2013 -0500 fix warnings in signal resource commit b1dd06da9ea7a094ed8bc9c762814bcd0c9b9f95 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Feb 25 15:13:10 2013 -0500 remove test resource from tosd directory commit 38a394072178d209698bbff7098b7798df5da1da Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Feb 25 10:16:34 2013 -0500 remove old mpi resource and test programs commit 0b0c63c5e1c3a04dd75867177bb39be75be659b3 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Feb 25 09:22:40 2013 -0500 remove development fprintfs commit 2f803762b771e16445f585092ecbe6e5f5edc9c5 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Feb 25 09:09:27 2013 -0500 cleaning up mpi method finalize a little commit fa770d06920573a985f03c4b8ac0d498eab2fde2 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Feb 25 08:53:55 2013 -0500 handle cancellation in mpi method commit b8adf3baf2a793c9ac8467f306c35aad15f0dcd1 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sun Feb 24 21:12:32 2013 -0500 reorganize finalize to avoid join race commit 5a9055acaed64915aa8c57f1371ba9793eb0347f Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sat Feb 23 22:09:38 2013 -0500 logic to adjust arrays after completion commit b723d36ae9f9669dd1fe91180ee83e763a18ae29 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sat Feb 23 21:59:38 2013 -0500 more fprintfs commit 6b764354123d0be6e45d7b82ff1f5a77fd33dc88 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sat Feb 23 21:32:16 2013 -0500 bug fix commit d5efde3fe249dcf1bfbda391f7fa48b09a392238 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sat Feb 23 21:21:03 2013 -0500 bug fixes and more fprintfs commit fb666bdba52c8e6e9cf39caacf074a8a1d85f1e0 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sat Feb 23 10:20:18 2013 -0500 bug fix, temporary debugging fprintfs commit 2706f45dfedd2e2681a384ac4bcca31c562dd7c4 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sat Feb 23 09:59:49 2013 -0500 mechanism to wake up MPI_Waitsome thread commit dcca227fff8ed6d6ddbaf9bb838be35ca46ed5cc Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sat Feb 23 09:44:51 2013 -0500 fill in partial logic for MPI_Waitsome commit dd11c94e0a7e8ce847c47869aa494004135d80b1 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sat Feb 23 08:01:04 2013 -0500 incremental progress commit 28498e8ddd3648ec920e45140fa2e7e7232ac618 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Feb 22 20:23:23 2013 -0500 incremental progress commit c7fcd1c76dd45e6a8f4dd1892acc47b3d0509dd6 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Feb 22 18:23:26 2013 -0500 incremental progress commit cff8da8f3dd9bd1042c5186f5094fd8df9449922 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Feb 22 17:38:31 2013 -0500 incremental progress commit 788b7996719e07084625c74d7b0d8e2bd7f970bb Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Feb 22 16:45:01 2013 -0500 start filling in data structures, init semaphores commit fbd05f69b1e78dfa5491e0b5c2d13aafba80f922 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Feb 22 16:16:55 2013 -0500 stub in worker thread commit 58a024d61c2c160edc1d517a87dfe5e70c805aaa Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Feb 22 15:42:53 2013 -0500 make comment blocks legible on 80 column terminals commit 11d14c984f1bf1802c38423160dd3f04af7ec9cb Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Feb 22 15:42:44 2013 -0500 turn on comment formatting commit f6725d88813bdcc68e9439bf3fd0680bcdae2fe4 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Feb 22 15:38:34 2013 -0500 try to fix some inconsistent code formatting commit 384eb8a8d8e5f34a5127859e720a534c41956ffc Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Feb 22 14:48:05 2013 -0500 update noop-bench to use aesop_main_set commit 0bccc904d4003091dcc838da9d8ad73cccf0e18d Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Feb 22 14:29:42 2013 -0500 convert rosd-create-bench to use aesop_main_set() commit 5f19463fed01222fea710abaa15bc867578bc5a6 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Feb 22 14:14:37 2013 -0500 update test program to aesop best practice commit 95e435078507581a5a89a835f92f6682703134c2 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Feb 22 13:59:57 2013 -0500 fix return code of cancel function commit cc756f4169d89b96bb1423feb4df3cd513cc4f5e Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Feb 22 13:58:24 2013 -0500 check for cancellation on post commit 33f43eb70ca8b5aeb7185aae30dd6e75e3f0c3f8 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Feb 22 13:56:22 2013 -0500 remove opcache, use ae_op_complete() commit b2c317e3312cc8f3aa022a364b78552b96398c0e Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Feb 22 13:48:06 2013 -0500 bug fixes and aesop best practice updates commit e3fe1ea751e508d895bf9459bd65b6f25d0c6d26 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Feb 21 09:56:24 2013 -0500 finish semaphore-based coalescing commit 34d1c3b0d85eb3b2d9b5c9d9209d508bcf63e48f Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Feb 21 09:08:16 2013 -0500 draft implementation of start_write() commit 4781317128829a9644670a319f23d1ba56b20d97 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Feb 20 21:48:26 2013 -0500 start stubbing in coalesce logic replacement - will use semaphore rather than sched resource when complete commit 6daf7e80591ad5a2224e2f5f7f4c070bf5fdcbda Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sun Feb 10 15:36:47 2013 -0500 update tosd-bench-create to new init scheme commit 8008ddc3ea282859a12206493cf0fbbdb0ed37b4 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sat Feb 9 20:20:43 2013 -0500 remove sqlite stuff from autoconf commit 35203361d9c861459a674ed5bc83ec10d457728c Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sat Feb 9 19:52:21 2013 -0500 explicitly specify -ldb for now commit 284561645540c1136190ccfb995acb8bad6428a8 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sat Feb 9 19:49:17 2013 -0500 remove bdb autoconf tests in Triton commit c806304331f2f8cb053a204ffbc9df225aef8957 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sat Feb 9 15:41:27 2013 -0500 remove deprecated bdb resource commit 1f703ce974599e3a7fd2757e37702f17c3d35545 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sat Feb 9 15:13:35 2013 -0500 remove deprecated bdb throttle code commit 64fb5f757cc03cf79de798750880221f37951fc9 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sat Feb 9 12:06:10 2013 -0500 convert tosd to use new aebdb interface commit 3d219f8cf93aae3cf0bb4da695412493f6a40718 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Feb 7 16:21:02 2013 -0500 remove deprecated triton version of sem resource commit 2b8947347e282bd9b45a121009b796eddc5f5817 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Feb 7 14:50:27 2013 -0500 get rid of mock network method commit a911ec69ab1c68e3131ed76580caf973b444aa1a Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Feb 5 16:46:59 2013 -0500 remove dbg comment from old formatting script commit f2e459d28d3992f68b89eca4c90c439c39db8dfc Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Jan 29 15:15:38 2013 -0500 clean up tosd fd cache remove function commit 8da4c851bf2d9be83e09dab747faf60c30bcb388 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Jan 28 10:56:17 2013 -0500 remove debugging assertion commit 8104e98dabb8a930d14cecec29b7d6055684f142 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Jan 28 10:54:49 2013 -0500 fix triton_buffer_t encoding bug - the encoder and decoder for triton_buffer_t did not match; it was encoding 32 bit values on the sender side and decoding 64 bit values on receiving side, eventually resulting in memory corruption commit 1e2b102b0648b5c84b3251cc98d99a1052cb4cd3 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Jan 28 10:45:51 2013 -0500 fix previous commit commit d2f45a287d6fb132fd25abb61c35e5de5776c55e Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Jan 28 10:20:14 2013 -0500 sanity check buffer allocation arguments commit d2c4357e74c72c1cb7965bb9f2cf96bf2fa7c433 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Jan 28 09:43:48 2013 -0500 avoid holding mutex across resource calls in tosd commit 265711f640df6b034dc5dc667b019fb02b8e1a85 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Jan 18 12:25:37 2013 -0500 workaround fix for decoding of triton_buffer_t - Don't shallow copy triton_buffer_t fields in rpc structs. For now we deep copy instead because of triton_buffer_t bugs in handling destruction of shallow copies. commit b366ae945af8c0d9f14d8d42bd96092d1372ba8f Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jan 17 15:27:17 2013 -0500 revert previous commit - This is making things worse (on the server side). Need to think about this some more. commit fe15e0eff88fe4eb866436ba66f3feb99ba94b89 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jan 17 14:49:59 2013 -0500 use aer init functions in rosd_read response commit 26d0d79250922978de785ebf4f7de78a60bafe7c Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Jan 11 17:24:11 2013 -0500 fix init ordering for rosd-create-bench commit d14be0a4b508f8b332ce8c39a8f045ced2d8c442 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Jan 11 17:23:40 2013 -0500 allow zeroconf values to be set before register commit 3ec6a6b4f85499c2068228acce5734bdfcf22e55 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Jan 11 16:56:29 2013 -0500 minor mutex fix commit 2a2127087a445e2490b2621ba8945c56551190ef Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Jan 11 15:19:17 2013 -0500 bug fix to ae function usage in tosd commit 98b7041fe37b6f4487c5341fd54ff804d6bb996e Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Jan 11 15:06:55 2013 -0500 fix initialization check in rosd commit d89cc95742455a3a8127191a961e5f61e30c0c73 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Jan 11 14:44:50 2013 -0500 bug fix - don't segfault trying to detect local address if some of the network methods are not initialized commit d4980485b24ae330892fb5d75b9f8e3cbdc86fc9 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jan 10 17:07:00 2013 -0500 updates to rosd-create-bench, not working yet commit 95813303214612e218da7a280240b6f26c4b9cec Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jan 10 17:06:45 2013 -0500 fixes to rosd initialization path commit 73bc6edc35daf90de6beeb4d702653f283560e0a Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Jan 10 15:10:08 2013 -0500 update init/finalize in rosd commit 73a4791e445a89239caf1dbefc592344238353ff Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Jan 7 17:33:30 2013 -0500 ref count for mapping finalize fn commit 6a4f0a2ae049c47856615963ac85c1cb6725776b Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Jan 7 17:29:27 2013 -0500 more init/finalize updates for noop-bench commit 6c5a6802603bc3f0f1caaf3fec80f4d2a35404a2 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Jan 7 16:54:52 2013 -0500 update init/finalize in state component commit 6db4dea3d93e6079ebc5f6c8dc1f3d4d43dbf972 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Jan 7 15:49:51 2013 -0500 update init/finalize for fakess commit 0b2b6c3237b755f011aa9f31801035d3df084ca3 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Jan 7 15:19:44 2013 -0500 bootstrapping for noop-bench commit 43ab97dc2ae9d5e1c3fc488ae1abdd5b5b64574b Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Jan 7 14:39:11 2013 -0500 safety checks commit f67d7cca178c3ecbd84d3c11d555d2be29fa61a3 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Jan 7 14:17:59 2013 -0500 misc updates - getting rpc stuff to initialize enough to run noop rpc functions commit 4715b4d15deb0caced8755cdb3ca0dadd01d5458 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Jan 7 11:47:09 2013 -0500 minor updates to noop bench commit 5dafb210496f8f1cac04eda653a82cd8951b1e80 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Jan 7 10:59:18 2013 -0500 update init/finalize for retry method commit 1b6da99fae20cc810922c7e36f8dd6c02c2e09fc Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sun Jan 6 21:15:27 2013 -0500 update init/finalize in message.ae - NOTE: there is probably going to be some sort of problem here from combining the stage1 and stage2 init functions. I'm not entirely sure why they were separated so I put them together for now. We'll figure it out later. commit 64ee3c7aa3a7b8fc2b4e575b12e8b2ec127eed3c Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sun Jan 6 20:39:15 2013 -0500 update init/finalize in service.ae commit c7290c917a113710c62c053546b972c1e2c0cb78 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Sun Jan 6 20:01:58 2013 -0500 update init/finalize in traffic cop commit 9bc8fcbc8e66c899f0e1122b1727c3f8099193d1 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Jan 4 17:16:18 2013 -0500 work around aer parser warnings commit 198df179b33be78c5858f4a38efea2ed2455d08c Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Jan 4 16:51:32 2013 -0500 update mpi-client-server test program - triton_msg api now working commit 5215f2b3927c060eb43a52049ef63b4c271e8942 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Jan 4 16:05:01 2013 -0500 revise initialization logic in mpi method commit 1f9a8b09bca4713c33d8e43a0ca7c2b713e2a57e Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Jan 4 14:07:29 2013 -0500 bug fix MPI finalization in pthread-compare test commit a3c58e0f7c7f742a20ca1bdaec5b9b6b848efa03 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Jan 4 14:06:32 2013 -0500 switch pthread-compare to use aesop_main_set commit bf9ba90c65e868aa6df94ab9e9ebd6ee761bbbf9 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Jan 4 11:53:45 2013 -0500 fix warnings in mpi resource commit 84afabb2ecb9e7536bb473200880467f4f3a5737 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Jan 4 11:28:16 2013 -0500 update initialization in mpi resource commit c0e7c1a67792a5b40e5fa32e4e84c631913783b6 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Jan 4 11:20:24 2013 -0500 revise triton-message initialization commit 74b8e33eec7df647fb7127de152e017d0cf10830 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 21 12:43:19 2012 -0500 note bug in test program commit 78345dba9a48194fc55fee8a5a9b299a67b6dc78 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 21 12:28:13 2012 -0500 bug fix to tosd_read commit d24812d9828c40287b63b2ba19ce0646702da769 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 21 11:36:38 2012 -0500 avoid using triton_init_check() commit acce22500c760ac4f05ee904495451918ab7a296 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 21 11:34:18 2012 -0500 force triton_init_check() failures for now commit ca22e856885df225d60c15d635def3793f9f2d15 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 21 11:29:57 2012 -0500 convert tosd1 to use aesop_main_set() commit 704f239c85d3f655318e480a19cdcd252d29ffef Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 21 11:24:29 2012 -0500 bug fix commit c794ec00acd38a023a93217c5415fffda842970a Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 21 11:14:21 2012 -0500 comments commit 1b1571861ea0e4e26d544c9707bd9e88faf39ade Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 21 11:13:11 2012 -0500 fix sched resource ref counting commit 8bc0cc630fccbb3a98e53514e26f8b1201ecc213 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 21 11:10:16 2012 -0500 fix warnings and types in sched resource commit dcbbeb6875a913c25b99f876a2761eac9f28be3b Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 21 10:32:26 2012 -0500 init/finalize debugging in tosd commit ac03db772abffef77278340f7cbb647a39d665af Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 21 10:29:41 2012 -0500 partial update to tosd1 for new init setup commit a78480552b35bf278e0a6254c0891579e48abe70 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 21 09:45:45 2012 -0500 fix tosd init setup commit efe9ed9084852fb6cf555dffdbef953a51dd4c6c Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 14 17:21:52 2012 -0500 remember to finalize triton-debug in bdb rsc commit ff3fc525a3cb52eb1c76c56fb7653ff97fa98832 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 14 17:20:24 2012 -0500 init zeroconf from triton-debug commit 7e1b4864f5b155f310b8291e8aa4da32828ead1a Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 14 17:18:19 2012 -0500 clean up init functions in zeroconf component commit ceb77281ded400639b264c6120e50f07a19f0872 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 14 17:10:25 2012 -0500 initialize triton debugging in bdb resource commit c4915a33ded97e9195a0217947c49c00661ccc72 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 14 17:08:42 2012 -0500 clean up init functinos for triton-debug commit eb9b77a5f3b0cd4a46af31bc4e2b72217ec72805 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 14 16:52:21 2012 -0500 fix bdb resource warnings commit f345b919dfa942c0c65dd78833431d7e4f52e050 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 14 15:56:15 2012 -0500 fix function prototypes in bdb-resource.c commit b6db762bccd8ec11b64fb9f990ef85c9e0d8d280 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 14 15:52:26 2012 -0500 fix bdb_init locking commit a161cdaaf59cb93639397e412a47c59b6f3b45dd Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 14 15:50:35 2012 -0500 full remove init registration from bdb resource commit 5cb630979958f7597c5a519092b304807a3aa6d7 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Dec 14 15:39:34 2012 -0500 add -lm commit c60548947419cc341a9721d883323bd0cca05177 Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Wed Dec 12 10:45:51 2012 -0700 Drop triton_init stuff commit f3ba350f019fbf96e959e1a64c6a6a0ace84f2fa Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Wed Dec 12 10:27:25 2012 -0700 Call map_init(). Does not work- triton_msg not initialized commit 2ec1f6a744d2725457110b4cbfdede2eb1eff3c3 Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Wed Dec 12 10:27:01 2012 -0700 Support VALGRIND commit 2bca828acbd4d1a6a8672eb040e16428c41e348e Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Wed Dec 12 10:16:49 2012 -0700 Drop unused definitions- they are in ae-remote-parser.h commit db0b17250a9a575ff550ef0237c6fabd1cdacab8 Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Wed Dec 12 10:16:30 2012 -0700 Add/fix note commit a1ac52436f16d4cc50bba045c3ab72b58ea070b2 Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Tue Dec 11 10:34:50 2012 -0700 Reference counting for mapping module commit 7e90e9cf54c030ffa3b32798634afba7f03ef68d Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Tue Dec 11 10:34:28 2012 -0700 Drop triton_socket commit b890719119356bc8aba031218f21cd0b7d9a7d32 Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Tue Dec 11 10:33:35 2012 -0700 Drop calls to init_register() commit 241ddd24c6ef88e69ae2fc86bc9d8da285fd1df5 Author: Dries Kimpe <dkimpe(a)mcs.anl.gov> Date: Wed Dec 5 16:07:02 2012 -0600 Add refcount to signal resource commit 3b3a6d8cbed6393050da3c513c20c5ede8abe013 Author: Dries Kimpe <dkimpe(a)mcs.anl.gov> Date: Wed Dec 5 15:56:19 2012 -0600 Add refcount init to bdb resource commit 9402a4fe854dfd2fadd74b15aab6a1838f426055 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Dec 3 20:22:47 2012 -0500 adjust aesop_main_set macros to match aesop/devel commit ca6afeac0f0f09e95f91b498fc8c3027bbdca4b6 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Dec 3 13:49:26 2012 -0500 remove aesop_cancel_branches_wait() macro commit f69c3ea83c2b61787ad1da1bb2110244d9c0217c Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Nov 30 11:12:12 2012 -0500 fix problem with missing encoder definitions commit eec62ba768a394eb3fbbc9d78285d1977a166056 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Nov 29 17:23:42 2012 -0500 port block-rmw to new thread resource commit 35476711c0a1c32ec71b14b6640210a93bfc7810 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Nov 29 17:20:19 2012 -0500 port phist-client to new thread resource - fix some unrelated warnings as well commit 6aa9a5e24d62f0d544ae81807669a3c2bfedca6b Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Nov 29 17:12:03 2012 -0500 warning note on documentation commit 4042a4ad0d54defd7ebe57d2cb2a4e912bf5166a Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Nov 29 17:08:42 2012 -0500 remove deprecated branch threader commit 3df28c6f96c10eb29fc0735d2c3f87d459cc6428 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Nov 29 16:56:37 2012 -0500 port tosd to new file resource commit a7a1d62507617c5e02fec900a2fba3d04ecd7e0a Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Nov 29 16:44:22 2012 -0500 port tosd-fd-cache to use new file resource commit e25190ef0d679c6d1cbc68755a2fd889d537a667 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Nov 29 16:36:58 2012 -0500 port aesop-mpi-bench to new file resource commit 17d505ee6b545a2c7eaba50e781940aa80150264 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Nov 29 16:29:03 2012 -0500 remove old file resource commit 14dbdecd63dc9fd4700ede163a83e31a1cd90b14 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Nov 29 16:28:03 2012 -0500 port echo-server to new file resource commit a01604d7df302b03e4765b68891cf194d8700a41 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Nov 29 15:38:55 2012 -0500 remove test-uint128 (now in c-utils) commit ad16f108b20cbb9b3d33b4a33c6616be47479242 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Nov 29 15:35:15 2012 -0500 remove testhash, now in c-utils repo commit 3273da951074d08a56930ede0f83f31579027104 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Nov 29 15:25:32 2012 -0500 removed testlist program (now in c-utils) commit 990b13ab9a94a16ba6e8f72abed5a42736dab92d Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Nov 29 14:19:54 2012 -0500 remove libev from triton tree commit 14fa99045432d8af4ae3efcdad6d46252d0ae29f Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Nov 29 14:15:33 2012 -0500 remove obsolete aesocket code - also port triton-socket to use new aesocket api - untested! commit 0a5706242d79476ff08df01ddfca3c64637289bc Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Nov 29 12:20:06 2012 -0500 remove duplicate test programs commit 7b37c2ef60c19dfe197093b23e82ee8a6c8f0b10 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Nov 29 12:16:47 2012 -0500 remove ae-blocking-parser from triton commit 6c362ec1dca946b3c20811e9917eeb5a3f761ceb Author: Phil Carns <carns(a)mcs.anl.gov> Date: Thu Nov 29 12:03:39 2012 -0500 remove test programs found in aesop repo - exception: parallelfor4 requires sched resource commit a5a78925c52a9835503d5e3ec5bd37e5800831fd Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Nov 21 15:37:20 2012 -0500 remove deprecated ae-ctl.c commit 3b89501d02d2f69f9b3aa957d21572663bb3ee42 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Nov 21 15:36:33 2012 -0500 remove deprecated aesop.c commit 4573e2bfa209ad15fc221ad2448b3f0e03ee190f Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Mon Nov 26 14:35:54 2012 -0700 Use aefile instead of Triton's file resource commit cf8b6ab3eef3062c8d25d277b95a10bc766a52ce Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Nov 21 15:28:15 2012 -0500 remove deprecated resource.c commit 6599ba42586b2af24d6cfb224ae07bff22df4ab1 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Nov 21 15:22:56 2012 -0500 remove deprecated aesop.h commit 8bafe4ccaacaf54a48f09afaaf15db27f2ee6cb3 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Nov 21 15:20:26 2012 -0500 remove deprecated resource.h commit 8e7ce3ed2fe5d1b67b8ad223c492d04e20205db3 Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Mon Nov 19 17:17:55 2012 -0700 Fix some undefined references commit b5806c8f920ef3f7a748f6c1ea6cbafc305a7175 Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Mon Nov 19 15:57:01 2012 -0700 Use arp_debug(), arp_trace() to perform conditional debugging output commit e7514bcb97d9fd1fc213d840a6547d9d9d54106a Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Mon Nov 19 15:56:11 2012 -0700 Clean up shell script debugging commit 95412f19a2315ba36d8018b3932a6891ee425714 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Nov 19 13:20:00 2012 -0500 remove some more deprecated headers commit fa2f219241d523da32556a77d622c416c15e5d5e Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Nov 19 11:33:04 2012 -0500 remove deprecated aesop cancel hack workaround commit a25b8430cf3d0cf665a025a6b425f6f2eac4cb1a Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Nov 19 11:31:39 2012 -0500 remove deprecated header commit 5dde73703b886d152e74f89a33ffa4e7817f2ebb Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Nov 19 11:30:08 2012 -0500 remove deprecated header commit 224e31605ae28f349e71e71584f048611cff9066 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Nov 19 11:28:50 2012 -0500 remove deprecated ae-base.h commit 59c5c7f1ead2c5f45ab281b39e3f773d46e40b18 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Nov 19 11:27:36 2012 -0500 remove op.h that has been moved to aesop commit ae72f8735d84fe11ff4cad5f671442d458a729d1 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Mon Nov 19 11:18:43 2012 -0500 switch to aesop timer - remove old triton timer directory and tests - switch triton_timer calls to aesop_timer calls commit c03bd68f7c98d99a09f4b184a34880af3251e5ab Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Nov 14 14:29:20 2012 -0500 remove headers and source dupulicated in c-utils - any divergence was merged into c-utils repo - all source updated to refer to copies available in <aesop/*> include directory that should be installed as part of aesop commit b32b61fd3b58d1b044bc7a758b21817621aafc32 Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Tue Nov 13 12:49:22 2012 -0700 Include path fixes commit aae8a41b367591ea5ffc8227c0b88da2480ac74b Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Tue Nov 13 12:47:43 2012 -0700 Restore aesop/parser/module.mk commit 8c5ffdb76e7af8d37cbfdaef40fc32a0b0dbdb5c Author: Justin M Wozniak <wozniak(a)mcs.anl.gov> Date: Wed Nov 7 15:21:10 2012 -0600 Changes for ticket #227 ----------------------------------------------------------------------- hooks/post-receive --

1 0

branch, master, updated. d2f3914898a9369aff41723834a8de70210e080c
by noreply＠mcs.anl.gov 29 Jul '13

29 Jul '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via d2f3914898a9369aff41723834a8de70210e080c (commit) via 33e2d9e2eca4b2c3f9dcdda72dbe938513eede9f (commit) from bedf8d515bd1c5ff78f4badbb0a47a7db612d271 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit d2f3914898a9369aff41723834a8de70210e080c Author: Cengiz <cengiz(a)Andrews-MacBook-Air.local> Date: Mon Jul 29 09:15:02 2013 -0400 Added vspace between subfigures in namespace use case commit 33e2d9e2eca4b2c3f9dcdda72dbe938513eede9f Author: Cengiz <cengiz(a)Andrews-MacBook-Air.local> Date: Mon Jul 29 07:50:35 2013 -0400 Minor modifications to namespace use case and Section II ----------------------------------------------------------------------- Summary of changes: papers/asg-osd/sections/namespace_usecase.tex | 9 +++++++-- papers/asg-osd/sections/proposed_model.tex | 21 ++++++++++----------- 2 files changed, 17 insertions(+), 13 deletions(-) Diff of changes: diff --git a/papers/asg-osd/sections/namespace_usecase.tex b/papers/asg-osd/sections/namespace_usecase.tex index df32e26..3a98c03 100644 --- a/papers/asg-osd/sections/namespace_usecase.tex +++ b/papers/asg-osd/sections/namespace_usecase.tex @@ -29,13 +29,13 @@ ASG entities are mapped to the namespace components as shown in Figure~\ref{fig: %\textcolor{red}{TODO: add a figure here showing the mappings more clearly} \begin{itemize} -\item Each record in ASG model corresponds to a key-value pair in a namespace. A key-value +\item Each record in ASG model corresponds to a key/value pair in a namespace. A key/value pair in a namespace corresponds to a directory entry. \item The identifier of each record in ASG corresponds an integer or string key in the namespace. \item Anything stored as the value of a namespace entry can be stored as the contents of an ASG record. -\item Each fork in ASG model corresponds to a collection of key-value pairs in a +\item Each fork in ASG model corresponds to a collection of key/value pairs in a namespace. \end{itemize} %One important feature required by the traditional @@ -57,30 +57,35 @@ except for the computation of integer hash of the string keys. \centering \fbox{\includegraphics[width=\columnwidth, keepaspectratio]{figs/integer-insert.pdf}} \caption{Insert} + \vspace{.9mm} \label{fig:integer-insert} \end{subfigure} \begin{subfigure}[b]{\columnwidth} \centering \fbox{\includegraphics[width=\columnwidth, keepaspectratio]{figs/integer-remove.pdf}} \caption{Remove} + \vspace{.9mm} \label{fig:integer-remove} \end{subfigure} \begin{subfigure}[b]{\columnwidth} \centering \fbox{\includegraphics[width=\columnwidth, keepaspectratio]{figs/integer-get.pdf}} \caption{Get} + \vspace{.9mm} \label{fig:integer-get} \end{subfigure} \begin{subfigure}[b]{\columnwidth} \centering \fbox{\includegraphics[width=\columnwidth, keepaspectratio]{figs/integer-list.pdf}} \caption{List} + \vspace{.9mm} \label{fig:integer-list} \end{subfigure} \begin{subfigure}[b]{\columnwidth} \centering \fbox{\includegraphics[width=\columnwidth, keepaspectratio]{figs/integer-rename.pdf}} \caption{Rename} + \vspace{.9mm} \label{fig:integer-rename} \end{subfigure} \begin{subfigure}[b]{\columnwidth} diff --git a/papers/asg-osd/sections/proposed_model.tex b/papers/asg-osd/sections/proposed_model.tex index 8ffc2f5..f9ebfe4 100644 --- a/papers/asg-osd/sections/proposed_model.tex +++ b/papers/asg-osd/sections/proposed_model.tex @@ -177,12 +177,12 @@ can be used to meet the requirements of common data models can be summarized as \textcolor{red}{TODO: examples for each} \begin{itemize} -\item\textbf{~\textit{Unified byte stream and key-value storage}}: -ASG storage model supports both byte-stream~\cite{Welch08scalableperformance} and key-value based storage. -Each byte is stored as -a one-byte record. With the numerical identifiers, each record can be also used as a key-value store. -As a result, ASG storage model supports both file based and key-value based access models and also -enables record oriented access for both of these models. +\item\textbf{~\textit{Unified byte stream and key/value storage}}: +ASG storage model supports both byte-stream~\cite{Welch08scalableperformance} and key/value based +storage~\cite{dynamodb}. Each byte is stored as a one-byte record. With the numerical identifiers +and record contents, each record can be also used as a key/value store. As a result, ASG storage +model supports both file based and key/value based access models and also enables record oriented +access for both of these models. \item\textbf{~\textit{Eliminating object attributes}}: Object-based storage models, such as T10~\cite{osd-3}, use attributes to describe the objects; meaning that object attributes are used to store metadata. In ASG storage model, we still have @@ -192,13 +192,12 @@ by reserved forks. Maintaining metadata in reserved forks gives opportunity to b stored in records and to meet the high performance requirement of common data models. This feature also makes it possible to move data closer to computation to preserve compute/storage locality. \item\textbf{~\textit{Record versioning}} -Versioning in ASG storage model enables sorting writes to a record as shown in previous studies~\cite{1251033, carns2010object}. -As the version number changes -with each write operation, highly concurrent write operations will be consistent and the performance -of the system will increase. +Versioning in ASG storage model enables sorting writes to a record as shown in previous +studies~\cite{1251033, carns2010object}. As the version number changes with each write operation, +highly concurrent write operations will be consistent and the performance of the system will increase. \item\textbf{~\textit{Conditional operations}} Conditional read and write operation flags provide synchronization primitives and atomicity in a data -model as it was previously shown in a couple of storage models~\cite{carns-pdsw12, simpledb, dynamodb, redis, hyperdex_atomic}, +model as it was previously shown in a couple of storage models~\cite{carns-pdsw12, simpledb, dynamodb, redis, hyperdex_atomic}. Using the conditional flags with ASG storage model operations, multiple processes can coordinate concurrent writes without using any explicit locking method. Using conditional flags also ensures that each ASG operation is completed with intermediate steps between their invocation and completion remaining hooks/post-receive --

1 0

branch, master, updated. bedf8d515bd1c5ff78f4badbb0a47a7db612d271
by noreply＠mcs.anl.gov 26 Jul '13

26 Jul '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via bedf8d515bd1c5ff78f4badbb0a47a7db612d271 (commit) from ebc9d1c76675afdacc7277bc87f6cc8e15a6b2ed (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit bedf8d515bd1c5ff78f4badbb0a47a7db612d271 Author: Cengiz <cengiz(a)Andrews-MacBook-Air.local> Date: Fri Jul 26 18:09:26 2013 -0400 Added citations for Section II-C, need some more citations * Last item, Server Location, needs some discussion ----------------------------------------------------------------------- Summary of changes: papers/asg-osd/paper.bib | 5 +++++ papers/asg-osd/sections/proposed_model.tex | 16 +++++++++++----- papers/asg-osd/sections/relatedwork.tex | 2 +- 3 files changed, 17 insertions(+), 6 deletions(-) Diff of changes: diff --git a/papers/asg-osd/paper.bib b/papers/asg-osd/paper.bib index aaf1dfe..292fa6c 100644 --- a/papers/asg-osd/paper.bib +++ b/papers/asg-osd/paper.bib @@ -626,3 +626,8 @@ title = {{Design of an Intelligent Object-based Storage device}}, month = {April}, year = {2013} } + +@misc{openvms-files-11, + title = {{HP OpenVMS Systems Documentation, Files-11 On Disk Structure Concepts}}, + url = {http://h71000.www7.hp.com/doc/731final/4506/4506pro\_001.html} +} diff --git a/papers/asg-osd/sections/proposed_model.tex b/papers/asg-osd/sections/proposed_model.tex index fc6da63..8ffc2f5 100644 --- a/papers/asg-osd/sections/proposed_model.tex +++ b/papers/asg-osd/sections/proposed_model.tex @@ -178,7 +178,8 @@ can be used to meet the requirements of common data models can be summarized as \textcolor{red}{TODO: examples for each} \begin{itemize} \item\textbf{~\textit{Unified byte stream and key-value storage}}: -ASG storage model supports both byte-stream and key-value based storage. Each byte is stored as +ASG storage model supports both byte-stream~\cite{Welch08scalableperformance} and key-value based storage. +Each byte is stored as a one-byte record. With the numerical identifiers, each record can be also used as a key-value store. As a result, ASG storage model supports both file based and key-value based access models and also enables record oriented access for both of these models. @@ -191,16 +192,19 @@ by reserved forks. Maintaining metadata in reserved forks gives opportunity to b stored in records and to meet the high performance requirement of common data models. This feature also makes it possible to move data closer to computation to preserve compute/storage locality. \item\textbf{~\textit{Record versioning}} -Versioning in ASG storage model enables sorting writes to a record. As the version number changes +Versioning in ASG storage model enables sorting writes to a record as shown in previous studies~\cite{1251033, carns2010object}. +As the version number changes with each write operation, highly concurrent write operations will be consistent and the performance of the system will increase. \item\textbf{~\textit{Conditional operations}} Conditional read and write operation flags provide synchronization primitives and atomicity in a data -model. Using the conditional flags with ASG storage model operations, multiple processes can coordinate +model as it was previously shown in a couple of storage models~\cite{carns-pdsw12, simpledb, dynamodb, redis, hyperdex_atomic}, +Using the conditional flags with ASG storage model operations, multiple processes can coordinate concurrent writes without using any explicit locking method. Using conditional flags also ensures that each ASG operation is completed with intermediate steps between their invocation and completion remaining unvisible to higher-level applications. \item\textbf{~\textit{Independently addressable records}} +ASG is a record-oriented storage model similar to some previous data models~\cite{openvms-files-11, Lovelace:2003:VD:1406022}. Each entity in ASG storage model has a numerical identifier and when ASG primitives access a record, they explicitly use the identifiers of the enclosing container, object and fork along with the identifier of that record. As a result, each record in ASG has a distinct location and they are independently accessable. @@ -208,8 +212,10 @@ Records are the smallest units of storage an operation can access in ASG model. addressable records makes it possible to support atomicity and concurrent read access since each record is independently accessable. \item\textbf{~\textit{Fork structure}} -Forks can be used to store metadata as discussed previously. In addition to that, they can also be used to -group records that store related data together. This approach enables to move related data closer to computation +Forks can be used to store metadata as discussed previously. In addition to that, it was shown that they can also +be used to +group records that store related data together ~\cite{russon2004ntfs, nieuwejaar:galley}. This approach enables +moving related data closer to computation to preserve compute/storage locality and to collect provenance from related records in an efficient way to support fault tolerance. \item\textbf{~\textit{Server location}} diff --git a/papers/asg-osd/sections/relatedwork.tex b/papers/asg-osd/sections/relatedwork.tex index 9bc08d5..2527888 100644 --- a/papers/asg-osd/sections/relatedwork.tex +++ b/papers/asg-osd/sections/relatedwork.tex @@ -29,7 +29,7 @@ Storage Targets (OSTs). %is responsible for handling metadata requests and making the metadata stored in MDTs available to the %clients; whereas OSS is responsible for filesystem I/O and for providing access to file data stored on %OSTs. -\textit{Ursa Minor} is a parallel file system that supports versioned writes. It keeps the existing +Ursa Minor~\cite{1251033} is a parallel file system that supports versioned writes. It keeps the existing object-storage interface~\cite{osd-3} mostly intact except for introducing \textit{slices} which are fragments of object data and it uses timestamps to distinguish different versions of data. Datamods~\cite{watkinsdatamods} is a framework that exploits existing large-scale storage system services hooks/post-receive --

1 0

C-Utils Repository branch, master, updated. 3aa9b75ce3e5bbbb06847ae3e3b598233fc53564
by noreply＠mcs.anl.gov 26 Jul '13

26 Jul '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "C-Utils Repository". The branch, master has been updated via 3aa9b75ce3e5bbbb06847ae3e3b598233fc53564 (commit) from e266e788b48b5196954cd62a8417866f1e26a893 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 3aa9b75ce3e5bbbb06847ae3e3b598233fc53564 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Jul 26 16:23:09 2013 -0400 fix --enable-valgrind-clean option ----------------------------------------------------------------------- Summary of changes: configure.ac | 4 +--- src/lookup3.c | 1 + 2 files changed, 2 insertions(+), 3 deletions(-) Diff of changes: diff --git a/configure.ac b/configure.ac index 4c476f2..4a6007b 100644 --- a/configure.ac +++ b/configure.ac @@ -68,11 +68,9 @@ AC_ARG_ENABLE(valgrind-clean, [Try harder to avoid valgrind warnings]) ]) -VALGRIND_CLEAN=0 AS_IF([test "x$enable_valgrind_clean" = "xyes"], [ - VALGRIND_CLEAN=1 + AC_DEFINE([VALGRIND], [1], [If enabling valgrind-clean build]) ]) -AC_SUBST(VALGRIND_CLEAN) dnl ========================================================================= dnl == Google's PerfTools =================================================== diff --git a/src/lookup3.c b/src/lookup3.c index bf76f5f..bfb6811 100644 --- a/src/lookup3.c +++ b/src/lookup3.c @@ -1,5 +1,6 @@ #include "jenkins-hash.h" +#include "c-utils-config.h" /* ------------------------------------------------------------------------------- hooks/post-receive -- C-Utils Repository

1 0

C-Utils Repository branch, master, updated. e266e788b48b5196954cd62a8417866f1e26a893
by noreply＠mcs.anl.gov 26 Jul '13

26 Jul '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "C-Utils Repository". The branch, master has been updated via e266e788b48b5196954cd62a8417866f1e26a893 (commit) from 127da042e1712a01c754559872b9c2d2f902cca9 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit e266e788b48b5196954cd62a8417866f1e26a893 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Jul 26 15:49:28 2013 -0400 fix warning ----------------------------------------------------------------------- Summary of changes: tests/test-uint128.c | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) Diff of changes: diff --git a/tests/test-uint128.c b/tests/test-uint128.c index 6508831..5d730b6 100644 --- a/tests/test-uint128.c +++ b/tests/test-uint128.c @@ -7,6 +7,7 @@ /* test program to exercies uint128_t utility functions */ #include <stdlib.h> +#include <string.h> #include "triton-uint128.h" #include "test-tap.h" hooks/post-receive -- C-Utils Repository

1 0

branch, master, updated. ebc9d1c76675afdacc7277bc87f6cc8e15a6b2ed
by noreply＠mcs.anl.gov 26 Jul '13

26 Jul '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via ebc9d1c76675afdacc7277bc87f6cc8e15a6b2ed (commit) via cd9355d9effba6171fad3543286a609420348297 (commit) from 853e184fca61c76199d1e40426171a6b26c4cdb1 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit ebc9d1c76675afdacc7277bc87f6cc8e15a6b2ed Author: Cengiz <cengiz(a)Andrews-MacBook-Air.local> Date: Fri Jul 26 08:06:10 2013 -0400 More modifications to Section II-C * Need to add examples to each feature to show that they are not new commit cd9355d9effba6171fad3543286a609420348297 Author: Cengiz <cengiz(a)Andrews-MacBook-Air.local> Date: Thu Jul 25 08:20:14 2013 -0400 Some initial work on the last section of proposed model ----------------------------------------------------------------------- Summary of changes: papers/asg-osd/sections/namespace_usecase.tex | 2 + papers/asg-osd/sections/proposed_model.tex | 47 +++++++++++++++++++++++++ 2 files changed, 49 insertions(+), 0 deletions(-) Diff of changes: diff --git a/papers/asg-osd/sections/namespace_usecase.tex b/papers/asg-osd/sections/namespace_usecase.tex index a4b35b3..df32e26 100644 --- a/papers/asg-osd/sections/namespace_usecase.tex +++ b/papers/asg-osd/sections/namespace_usecase.tex @@ -47,6 +47,8 @@ The namespace implementation has \textit{insert}, \textit{remove}, Thse operations are mostly POSIX-compliant and they call underlying ASG storage model primitives, defined in Section \ref{primitives}, in order to accomplish their functionalities. Namespace operations for integer keys are shown in Figure~\ref{fig:integer-ops}. +Namespace operations for string keys are mostly similar to the namespace operations for integer keys +except for the computation of integer hash of the string keys. \textcolor{red}{TODO: should we insert a figure describing string key operations?} \begin{figure}[!t] diff --git a/papers/asg-osd/sections/proposed_model.tex b/papers/asg-osd/sections/proposed_model.tex index ae8f2b4..fc6da63 100644 --- a/papers/asg-osd/sections/proposed_model.tex +++ b/papers/asg-osd/sections/proposed_model.tex @@ -167,3 +167,50 @@ thoughts about minimizing complexity: you don't have special attribute API with the rest of the API, you don't have special atomic modes; all access is atomic. Records are first-class citizens, you don't have to translate from records into byte streams, etc.} + +In this section, we introduce the features provided by the ASG storage model and +explain how they make it possible to meet the requirements of the common data models +listed in Table~\ref{tbl:reqs}. It is important to note that none of these features are new; +ASG storage model just presents a reusable unified API bringing these features together +while minimizing complexity. The features provided by the ASG storage model and how they +can be used to meet the requirements of common data models can be summarized as follows; + +\textcolor{red}{TODO: examples for each} +\begin{itemize} +\item\textbf{~\textit{Unified byte stream and key-value storage}}: +ASG storage model supports both byte-stream and key-value based storage. Each byte is stored as +a one-byte record. With the numerical identifiers, each record can be also used as a key-value store. +As a result, ASG storage model supports both file based and key-value based access models and also +enables record oriented access for both of these models. +\item\textbf{~\textit{Eliminating object attributes}}: +Object-based storage models, such as T10~\cite{osd-3}, use attributes to describe the objects; +meaning that object attributes are used to store metadata. In ASG storage model, we still have +metadata describing the records; however we do not store them in separate attributes as it is +normally done in object-based storage models. The functionality of describing records can be provided +by reserved forks. Maintaining metadata in reserved forks gives opportunity to better scale data +stored in records and to meet the high performance requirement of common data models. This feature also +makes it possible to move data closer to computation to preserve compute/storage locality. +\item\textbf{~\textit{Record versioning}} +Versioning in ASG storage model enables sorting writes to a record. As the version number changes +with each write operation, highly concurrent write operations will be consistent and the performance +of the system will increase. +\item\textbf{~\textit{Conditional operations}} +Conditional read and write operation flags provide synchronization primitives and atomicity in a data +model. Using the conditional flags with ASG storage model operations, multiple processes can coordinate +concurrent writes without using any explicit locking method. Using conditional flags also ensures that +each ASG operation is completed with intermediate steps between their invocation and completion remaining +unvisible to higher-level applications. +\item\textbf{~\textit{Independently addressable records}} +Each entity in ASG storage model has a numerical identifier and when ASG primitives access a record, they +explicitly use the identifiers of the enclosing container, object and fork along with the identifier of +that record. As a result, each record in ASG has a distinct location and they are independently accessable. +Records are the smallest units of storage an operation can access in ASG model. Having access to independently +addressable records makes it possible to support atomicity and concurrent read access since each record is +independently accessable. +\item\textbf{~\textit{Fork structure}} +Forks can be used to store metadata as discussed previously. In addition to that, they can also be used to +group records that store related data together. This approach enables to move related data closer to computation +to preserve compute/storage locality and to collect provenance from related records in an efficient way to +support fault tolerance. +\item\textbf{~\textit{Server location}} +\end{itemize} hooks/post-receive --

1 0

branch, master, updated. 853e184fca61c76199d1e40426171a6b26c4cdb1
by noreply＠mcs.anl.gov 24 Jul '13

24 Jul '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 853e184fca61c76199d1e40426171a6b26c4cdb1 (commit) from 7bd247cd3da674b668b6dc019a861ba54654be8a (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 853e184fca61c76199d1e40426171a6b26c4cdb1 Author: Cengiz <cengiz(a)Andrews-MacBook-Air.local> Date: Wed Jul 24 23:19:57 2013 -0400 Added a subfigure for update operation too * Should we also add a figure for string key operations? ----------------------------------------------------------------------- Summary of changes: papers/asg-osd/figs/integer-update.pdf | Bin 0 -> 11061 bytes papers/asg-osd/figs/integer-update.svg | 289 +++++++++++++++++++++++++ papers/asg-osd/sections/namespace_usecase.tex | 21 ++- 3 files changed, 302 insertions(+), 8 deletions(-) create mode 100644 papers/asg-osd/figs/integer-update.pdf create mode 100644 papers/asg-osd/figs/integer-update.svg Diff of changes: diff --git a/papers/asg-osd/figs/integer-update.pdf b/papers/asg-osd/figs/integer-update.pdf new file mode 100644 index 0000000..fa47728 Binary files /dev/null and b/papers/asg-osd/figs/integer-update.pdf differ diff --git a/papers/asg-osd/figs/integer-update.svg b/papers/asg-osd/figs/integer-update.svg new file mode 100644 index 0000000..2e490dc --- /dev/null +++ b/papers/asg-osd/figs/integer-update.svg @@ -0,0 +1,289 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> + + +<svg + xmlns:dc="http://purl.org/dc/elements/1.1/" + xmlns:cc="http://creativecommons.org/ns#" + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" + xmlns:svg="http://www.w3.org/2000/svg" + xmlns="http://www.w3.org/2000/svg" + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd" + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape" + width="744.09448819" + height="1052.3622047" + id="svg2" + version="1.1" + inkscape:version="0.48.2 r9819" + sodipodi:docname="integer-update.pdf"> + <defs + id="defs4"> + <marker + inkscape:stockid="Arrow2Lend" + orient="auto" + refY="0.0" + refX="0.0" + id="Arrow2Lend" + style="overflow:visible;"> + <path + id="path3892" + style="font-size:12.0;fill-rule:evenodd;stroke-width:0.62500000;stroke-linejoin:round;" + d="M 8.7185878,4.0337352 L -2.2072895,0.016013256 L 8.7185884,-4.0017078 C 6.9730900,-1.6296469 6.9831476,1.6157441 8.7185878,4.0337352 z " + transform="scale(1.1) rotate(180) translate(1,0)" /> + </marker> + </defs> + <sodipodi:namedview + id="base" + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1.0" + inkscape:pageopacity="0.0" + inkscape:pageshadow="2" + inkscape:zoom="1.7915184" + inkscape:cx="303.70605" + inkscape:cy="952.56961" + inkscape:document-units="px" + inkscape:current-layer="layer1" + showgrid="true" + inkscape:window-width="1440" + inkscape:window-height="765" + inkscape:window-x="0" + inkscape:window-y="0" + inkscape:window-maximized="0" + inkscape:snap-global="false" + showguides="false"> + <inkscape:grid + type="xygrid" + id="grid2985" /> + </sodipodi:namedview> + <metadata + id="metadata7"> + <rdf:RDF> + <cc:Work + rdf:about=""> + <dc:format>image/svg+xml</dc:format> + <dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage" /> + <dc:title></dc:title> + </cc:Work> + </rdf:RDF> + </metadata> + <g + inkscape:label="Layer 1" + inkscape:groupmode="layer" + id="layer1"> + <g + id="g2995" + transform="translate(-20,0)"> + <text + sodipodi:linespacing="125%" + id="text2987" + y="43.831421" + x="69.774414" + style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + xml:space="preserve"><tspan + style="font-size:14px;text-align:center;text-anchor:middle" + y="43.831421" + x="69.774414" + id="tspan2989" + sodipodi:role="line">Integer key</tspan><tspan + style="font-size:14px;text-align:center;text-anchor:middle" + id="tspan2991" + y="61.331421" + x="69.774414" + sodipodi:role="line">Write buffer</tspan></text> + <rect + y="27.362183" + x="25" + height="40" + width="90" + id="rect2993" + style="fill:none;stroke:#000000;stroke-width:0.963;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;stroke-dashoffset:0" /> + </g> + <text + xml:space="preserve" + style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="129.87207" + y="51.029663" + id="text3020" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan3022" + x="129.87207" + y="51.029663" + style="font-size:14px">probe_fork(location, key, intermediate info buffer, 1, NULL) </tspan></text> + <rect + style="fill:none;stroke:#000000;stroke-width:0.963;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;stroke-dashoffset:0" + id="rect3026" + width="425" + height="20" + x="120" + y="37.36219" /> + <g + id="g3040" + transform="translate(-2.5,-0.00457129)"> + <text + sodipodi:linespacing="125%" + id="text3028" + y="101.33142" + x="334.81543" + style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + xml:space="preserve"><tspan + style="font-size:14px;text-align:center;text-anchor:middle" + y="101.33142" + x="334.81543" + id="tspan3030" + sodipodi:role="line">Given key</tspan><tspan + style="font-size:14px;text-align:center;text-anchor:middle" + id="tspan3032" + y="118.83142" + x="334.81543" + sodipodi:role="line">exists?</tspan></text> + <path + d="m 385,104.86218 c 0,12.42641 -22.38576,22.5 -50,22.5 -27.61424,0 -50,-10.07359 -50,-22.5 0,-12.426404 22.38576,-22.499997 50,-22.499997 27.61424,0 50,10.073593 50,22.499997 z" + sodipodi:ry="22.5" + sodipodi:rx="50" + sodipodi:cy="104.86218" + sodipodi:cx="335" + id="path3038" + style="fill:none;stroke:#000000;stroke-width:0.963;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;stroke-dashoffset:0" + sodipodi:type="arc" /> + </g> + <g + id="g3052" + transform="translate(0,2.5)"> + <text + sodipodi:linespacing="125%" + id="text3046" + y="106.18347" + x="421.021" + style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + xml:space="preserve"><tspan + style="font-size:14px" + y="106.18347" + x="421.021" + id="tspan3048" + sodipodi:role="line">no</tspan></text> + <rect + y="92.362183" + x="415" + height="20" + width="30" + id="rect3050" + style="fill:none;stroke:#000000;stroke-width:0.963;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;stroke-dashoffset:0" /> + </g> + <g + id="g3063" + transform="translate(-95,-47.5)"> + <text + sodipodi:linespacing="125%" + id="text3057" + y="154.82654" + x="318.03711" + style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + xml:space="preserve"><tspan + style="font-size:14px" + y="154.82654" + x="318.03711" + id="tspan3059" + sodipodi:role="line">yes</tspan></text> + <rect + y="142.36218" + x="315" + height="20" + width="30" + id="rect3061" + style="fill:none;stroke:#000000;stroke-width:0.963;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;stroke-dashoffset:0" /> + </g> + <g + id="g3076" + transform="translate(1.4282227,3.7880993)"> + <text + sodipodi:linespacing="125%" + id="text3068" + y="162.36218" + x="65" + style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + xml:space="preserve"><tspan + style="font-size:14px" + y="162.36218" + x="65" + id="tspan3070" + sodipodi:role="line">write(location, key, write buffer, ALL, intermediate info buffer->version + 1)</tspan></text> + <rect + y="148.47699" + x="61.071777" + height="20.045786" + width="540" + id="rect3074" + style="fill:none;stroke:#000000;stroke-width:0.78718573;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;stroke-dashoffset:0" /> + </g> + <g + id="g3092" + transform="translate(-7.0561523,2.5)"> + <g + id="g3087"> + <text + xml:space="preserve" + style="font-size:40px;font-style:normal;font-weight:normal;line-height:125%;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;font-family:Sans" + x="487.05615" + y="107.58142" + id="text3081" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan3083" + x="487.05615" + y="107.58142" + style="font-size:14px">end</tspan></text> + <rect + style="fill:none;stroke:#000000;stroke-width:0.963;stroke-miterlimit:4;stroke-opacity:1;stroke-dasharray:none;stroke-dashoffset:0" + id="rect3085" + width="30" + height="20" + x="485" + y="92.362183" /> + </g> + </g> + <path + style="fill:none;stroke:#000000;stroke-width:0.96470338px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow2Lend)" + d="m 94.992454,47.379176 23.220686,0" + id="path3098" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.97352666px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow2Lend)" + d="m 332.49994,57.527515 0,23.10809" + id="path4312" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.98661262px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow2Lend)" + d="m 234.99997,114.93393 0,35.58338" + id="path4706" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:1px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow2Lend)" + d="m 492.78528,152.06636 0.25254,-35.35534" + id="path4914" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.98585331px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow2Lend)" + d="m 282.3285,104.84025 -30.66385,0" + id="path5100" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.98200285px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow2Lend)" + d="m 382.75886,104.81622 30.49579,0.0895" + id="path5284" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + <path + style="fill:none;stroke:#000000;stroke-width:0.98111594px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1;marker-end:url(#Arrow2Lend)" + d="m 445.07985,104.87947 31.1232,0" + id="path5468" + inkscape:connector-type="polyline" + inkscape:connector-curvature="3" /> + </g> +</svg> diff --git a/papers/asg-osd/sections/namespace_usecase.tex b/papers/asg-osd/sections/namespace_usecase.tex index 35ab1e1..a4b35b3 100644 --- a/papers/asg-osd/sections/namespace_usecase.tex +++ b/papers/asg-osd/sections/namespace_usecase.tex @@ -47,41 +47,46 @@ The namespace implementation has \textit{insert}, \textit{remove}, Thse operations are mostly POSIX-compliant and they call underlying ASG storage model primitives, defined in Section \ref{primitives}, in order to accomplish their functionalities. Namespace operations for integer keys are shown in Figure~\ref{fig:integer-ops}. -\textcolor{red}{TODO: maybe insert figures describing how the namespace operations -are using underlying ASG primitives} +\textcolor{red}{TODO: should we insert a figure describing string key operations?} -\begin{figure}[t] +\begin{figure}[!t] \centering \begin{subfigure}[b]{\columnwidth} \centering - \includegraphics[width=\columnwidth, keepaspectratio]{figs/integer-insert.pdf} + \fbox{\includegraphics[width=\columnwidth, keepaspectratio]{figs/integer-insert.pdf}} \caption{Insert} \label{fig:integer-insert} \end{subfigure} \begin{subfigure}[b]{\columnwidth} \centering - \includegraphics[width=\columnwidth, keepaspectratio]{figs/integer-remove.pdf} + \fbox{\includegraphics[width=\columnwidth, keepaspectratio]{figs/integer-remove.pdf}} \caption{Remove} \label{fig:integer-remove} \end{subfigure} \begin{subfigure}[b]{\columnwidth} \centering - \includegraphics[width=\columnwidth, keepaspectratio]{figs/integer-get.pdf} + \fbox{\includegraphics[width=\columnwidth, keepaspectratio]{figs/integer-get.pdf}} \caption{Get} \label{fig:integer-get} \end{subfigure} \begin{subfigure}[b]{\columnwidth} \centering - \includegraphics[width=\columnwidth, keepaspectratio]{figs/integer-list.pdf} + \fbox{\includegraphics[width=\columnwidth, keepaspectratio]{figs/integer-list.pdf}} \caption{List} \label{fig:integer-list} \end{subfigure} \begin{subfigure}[b]{\columnwidth} \centering - \includegraphics[width=\columnwidth, keepaspectratio]{figs/integer-rename.pdf} + \fbox{\includegraphics[width=\columnwidth, keepaspectratio]{figs/integer-rename.pdf}} \caption{Rename} \label{fig:integer-rename} \end{subfigure} +\begin{subfigure}[b]{\columnwidth} + \centering + \fbox{\includegraphics[width=\columnwidth, keepaspectratio]{figs/integer-update.pdf}} + \caption{Update} + \label{fig:integer-update} +\end{subfigure} \caption{Namespace operations using integer keys} \label{fig:integer-ops} \end{figure} hooks/post-receive --

1 0