February 2012 - Triton-commits - mailman3.cels.anl.gov

Triton Repository branch, master, updated. 155501673886427f5eb28b0eb82be2aecbc4dc7b
by noreply＠mcs.anl.gov 28 Feb '12

28 Feb '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton Repository". The branch, master has been updated via 155501673886427f5eb28b0eb82be2aecbc4dc7b (commit) via 8308e7466aaa9dbd45bf981ae4c522d22fd4bbf1 (commit) from 69395960a5db99a0607dbc4e88ca97d868848490 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 155501673886427f5eb28b0eb82be2aecbc4dc7b Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Feb 28 10:18:53 2012 -0500 edits and comments from RobR commit 8308e7466aaa9dbd45bf981ae4c522d22fd4bbf1 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Tue Feb 28 10:18:26 2012 -0500 edits and comments from RobR ----------------------------------------------------------------------- Summary of changes: code/doc/{resilience => }/fault-injection.txt | 0 code/doc/module.mk.in | 3 +- code/doc/resilience/client-fault-handling.txt | 10 ++- code/doc/resilience/failure-detection.txt | 43 ++++++++++--- code/doc/resilience/module.mk.in | 1 - code/doc/resilience/non-idempotent-retry.txt | 14 +++-- code/doc/resilience/object-placement.txt | 26 +++++++- code/doc/resilience/object-rebuild.txt | 54 +++++++++++------ code/doc/resilience/resilience-book.txt.in | 10 +-- code/doc/resilience/resiliency-overview.txt | 7 +-- code/doc/resilience/rosd-replication.txt | 78 +++++++++++++++++-------- 11 files changed, 167 insertions(+), 79 deletions(-) rename code/doc/{resilience => }/fault-injection.txt (100%) Diff of changes: diff --git a/code/doc/resilience/fault-injection.txt b/code/doc/fault-injection.txt similarity index 100% rename from code/doc/resilience/fault-injection.txt rename to code/doc/fault-injection.txt diff --git a/code/doc/module.mk.in b/code/doc/module.mk.in index 49b4a92..3f80ac9 100644 --- a/code/doc/module.mk.in +++ b/code/doc/module.mk.in @@ -4,4 +4,5 @@ DOCSRC += $(DIR)/test.txt \ $(DIR)/pipelining.txt \ $(DIR)/compiling-triton.txt \ $(DIR)/aesop.txt \ - $(DIR)/aesop-pbranch-cancel.txt + $(DIR)/aesop-pbranch-cancel.txt \ + $(DIR)/fault-injection.txt diff --git a/code/doc/resilience/client-fault-handling.txt b/code/doc/resilience/client-fault-handling.txt index da22561..cfddd56 100644 --- a/code/doc/resilience/client-fault-handling.txt +++ b/code/doc/resilience/client-fault-handling.txt @@ -27,11 +27,12 @@ recovery mechanisms. The client can query any server at any time to retrieve a snapshot of the current state of the system that indicates which servers are available. The client will use this information, in conjunction with an algorithmic -placement function, to map object IDs to servers. See object-placement.txt +placement function, to map object IDs to servers and to make decisions on +object placement. See object-placement.txt for more details. A client will assume that it's snapshot of the system state is correct -until it either encounters a failure (i.e., a timeout communicating with a +unless it either encounters a failure (i.e., a timeout communicating with a specific server) or else receives an error code from a server indicating that it has contacted the wrong server. The client will then refresh its state information by requesting an update from a functioning server. @@ -151,8 +152,9 @@ into each RPC) to correlate high level client operations to RPCs. Note that if the client keeps receiving the same system state information each time that it requests a copy from the servers, then the system -has reached a quiescent state (ie, it is detecting no new failures). +has reached a quiescent state (i.e., it is detecting no new failures). If operations continue to fail with retryable errors despite the fact that the system is in a quiescent state, then the client library must eventually -report a failure to the application. +report a failure to the application. This would be indicative of a software +failure. diff --git a/code/doc/resilience/failure-detection.txt b/code/doc/resilience/failure-detection.txt index 9763a12..5faf6e6 100644 --- a/code/doc/resilience/failure-detection.txt +++ b/code/doc/resilience/failure-detection.txt @@ -24,16 +24,17 @@ stateless. There is therefore no need for the servers to pro-actively detect cl failures. There is also no need for clients to detect the failure of other clients. -* Failure detection must scale to thousands of server nodes without a -significant increase in either detection or dissemination time. +* Failure detection must scale to large numbers of nodes without a +significant increase in either detection or dissemination time. Linear +scaling of detection time is not acceptable. * If failure detection protocol requires periodic exchange of messages between servers, then we will attach extra information to these messages -(ie, piggy-backing) in order to publish and disseminate +(i.e., piggy-backing) in order to publish and disseminate system-specific state information among servers that extends beyond simple "up" and "down" status values. This will give each server the ability to construct a relatively recent snapshot of the overall system state in -terms of several parameters such as load or capacity. +terms of several parameters, such as load or capacity. * We must be able to calculate an upper bound on failure detection and dissemination for any given number of servers. This information will be used to @@ -42,11 +43,24 @@ timeout values. * We must handle the case of multiple concurrent failures. +* We must gracefully handle network partitions, in which no servers have +failed but a subset of servers have lost connectivity to the other servers +in the system. In this scenario the system must at least maintain +availability for the majority partition while preventing the minority partition +from servicing requests. + * We _do not_ need strong ordering of failure detection. In other words, if server A fails before server B then it is still acceptable for server C to find out -about those failures in the reverse order as long as both notifications +about those failures in the reverse order, as long as both notifications occur within a bounded amount of time. +It is possible that future systems or system software will provide some form +of "resilience backplane" capable of reporting error conditions. The Triton +design does not rely on this functionality, but the presence of such a +component would simplify failure detection significantly. The remainder of +this document operates on the assumption that there is no resilience +backplane available in the system. + === Failure Detection Background and Terminology Failure detectors generally consist of two parts: @@ -59,7 +73,7 @@ failure detection communication among a collection of nodes The traditional mechanism for failure detection is to send _heartbeat_ messages. Heartbeat relies on regularly _telling_ other nodes that you're still alive. If a heartbeat message is not received from a node -within a given time period the the node is assumed to have failed. An alternative is to +within a given time period, then the the node is assumed to have failed. An alternative is to _ping_ other nodes, by explicitly asking them if they are still alive, and if they don't acknowledge, they are assumed to have failed. Intuitively, heartbeat only requires a single message sent to indicate liveness, @@ -175,7 +189,7 @@ system state information with minimal overhead. The *system state* component of each Triton server will be responsible for detecting failures, maintaining group membership, and disseminating general system state information. It will automatically communicate -with other servers to maintain an up-to-date view of the system as +with other servers to maintain an relatively up-to-date view of the system as a whole on each server. The data maintained on a given server can be queried by other components at any time without triggering any additional explicit communication. @@ -184,7 +198,9 @@ The interface to the system state component will allow callers to obtain a list of servers in the system along with their current status. The status will include simple "up" and "down" states, as well as finer-grained states to indicate when a server is rebuilding or attempting to join -the system. Published state information for a given server can be +the system. If the number of servers is large, then the interface will also +allow callers to retrieve subsets of servers based on ID ranges in order to +limit the scope of the data. Published state information for a given server can be queried from the system state component using a key/value interface. See the placement document for information on how data provided by the system state component is used by the placement algorithm. @@ -268,7 +284,7 @@ automatically derive appropriate timeout values for various scenarios. The system will start with conservative estimates as initial conditions and update those values over time as more information is gathered about the -storage system. +storage system, allowing faster convergence to consensus. === Failure actions @@ -301,7 +317,7 @@ else request updated system state information from another server. ==== Rebuilding: -The peer servers for a failed server (ie, servers within N distance of the +The peer servers for a failed server (i.e., servers within N distance of the failed server according to the placement algorithm, where N is the maximum replication factor) must take action to rebuild missing objects following the failure of a server. However, once a peer has detected a failure, it must wait _at @@ -319,6 +335,13 @@ code to clients). Triton will not allow writes to proceed until there is a consensus on the ownership of a given object. However, it can allow read access to those objects. +Note that the behavior described above (blocking write access for a period +of time until consensus is achieved) will only impact a subset of servers in +the system. Servers that are not participating in a rebuild will be +unaffected. Our expectation is that higher-level softare layers could work +around affected servers by selecting different OID regions when writing new +objects. + == Related work in file system failure detection === Ceph diff --git a/code/doc/resilience/module.mk.in b/code/doc/resilience/module.mk.in index 0ca79dd..a312df1 100644 --- a/code/doc/resilience/module.mk.in +++ b/code/doc/resilience/module.mk.in @@ -2,7 +2,6 @@ DIR := doc/resilience DOCSRC += \ $(DIR)/resiliency-overview.txt \ - $(DIR)/fault-injection.txt \ $(DIR)/rosd-replication.txt \ $(DIR)/object-rebuild.txt \ $(DIR)/client-fault-handling.txt \ diff --git a/code/doc/resilience/non-idempotent-retry.txt b/code/doc/resilience/non-idempotent-retry.txt index db3264a..3b3db63 100644 --- a/code/doc/resilience/non-idempotent-retry.txt +++ b/code/doc/resilience/non-idempotent-retry.txt @@ -113,8 +113,8 @@ used in DAFS <<dafs>>. === Lustre A Lustre <<Schwan>> file system consists of a set of object storage servers -and a metadata server. All object storage server operations are -idempotent, while the metadata server provides a mechanism to safely retry NI +and a metadata server. The metadata server is authoritative in terms of the +state of user-accessible files and directories provides a mechanism to safely retry NI operations. Clients assign unique transaction identifier to each metadata request. The metadata server stores this identifier as well as relevant data from the response persistently on disk @@ -134,7 +134,7 @@ client is actually a proxy for multiple independent processes. This scenario may arise when multiple cores or processes on a node share a single client instance or if I/O function calls are forwarded from lightweight compute node kernels to intermediate gateway nodes. As an example of the latter -case, the current generation IBM BlueGene/P systems use I/O forwarding +case, the IBM BlueGene/P systems use I/O forwarding nodes that each service I/O operations from many compute nodes and compute cores through a single file system client interface. @@ -203,7 +203,7 @@ non-idempotent remote storage operations with the following properties: The triton storage system is built atop a local object storage abstraction known as the Transactional Object Storage Device (TOSD). We propose that this object storage model be augmented to associate an additional parameter, known as the non-idempotent identifier, with each -object operation. This non-idempotent identifier (NIID) is an opaque 64-bit +non-idempotent object operation. This non-idempotent identifier (NIID) is an opaque 64-bit integer that uniquely identifies the client request that triggered the operation. The NIID may be set to zero for operations that are considered to be idempotent by both the file system and the @@ -250,7 +250,7 @@ These idle periods can be used for any automatic reclamation activities. === Protocol The basic protocol for client RPCs is similar to that employed by -existing systems. Each request generated by the client is assigned a new +existing systems. Each non-idempotent request generated by the client is assigned a new transaction ID. If the request fails, then the same request is resent using the same transaction ID. The retry policy and retry timeout is up to the system implementer and has no impact on how NI operations are @@ -271,7 +271,9 @@ latexmath:[sizeof(OID/NIID values) * num\_servers * max\_concurrent]. This approach is similar to that taken by Ceph, but it relaxes the requirement that transaction IDs be assigned sequentially. Requests can be stalled for an arbitrarily long period of time without impeding the reclamation of -subsequent transaction IDs. +subsequent transaction IDs. This aggressive approach to reclamation will +most likely be relevant primarily for object delete operations which have the +potential to leave shadow objects on the system. == Bibliography diff --git a/code/doc/resilience/object-placement.txt b/code/doc/resilience/object-placement.txt index 4354bd7..8fe776a 100644 --- a/code/doc/resilience/object-placement.txt +++ b/code/doc/resilience/object-placement.txt @@ -84,12 +84,22 @@ retain authoritative ability to overrule a client. The trade-off for any deterministic placement system is that once an object has been assigned an identifier and stored on the system, there is no -particularly practical way to move it. We will attempt to make intelligent +particularly practical way to move it from a storage system point of view. We will attempt to make intelligent placement decisions at object creation time. However, if an object (or set of objects) is found to be in a suboptimal location later due to changing system parameters or changing workloads, then we will explicitly reorganize the objects by selecting new object identifiers and transferring data as -needed. This approach will be explored as a long term research avenue. +needed via support in higher-level software. +This approach will be explored as a long term research avenue. The +prototype will focus on the common case of static object mapping. + +To summarize the object placement strategy of Triton, any entity in the +system (client or server) will be capable of mapping arbitrary objects to +appropriate servers using a known algorithm with no explicit communication. +This algorithm will also allow any client or server to deterministically +idenitify affected servers following a failure. Failure events +will therefore have a deterministic, well-known impact on the system in that +regard. === Implications for rebuild and replication @@ -126,7 +136,17 @@ latexmath:[$ {\cal F}(O, \{N_1,N_2,N_3,...\}) \rightarrow \{M=N_i, R_1=N_j, R_2= R_3=N_l, ...\} $] This allows us to also define a _replication group_, consisting -of nodes that a node latexmath:[$N_i$] shares objects with through replication. +of nodes that a node latexmath:[$N_i$] shares objects with through +replication. Clients can select any node in a replica group to read data +from. We expect that higher level software may leverage this functionality +to increase bandwidth in the read case by leveraging as many copies of data +as possible. For consistency reasons (to be defined in +rosd-replication.txt) only the first server in the replication group will accept +write operations, however. + +Servers will use the same mapping function as clients in order to confirm +the placement consistency for incoming requests and to identify peers for +replication purposes. == Bibliography diff --git a/code/doc/resilience/object-rebuild.txt b/code/doc/resilience/object-rebuild.txt index 649537d..85afba6 100644 --- a/code/doc/resilience/object-rebuild.txt +++ b/code/doc/resilience/object-rebuild.txt @@ -12,10 +12,10 @@ case. == Overview -Each Triton server will monitor its N*2 nearest neighbor servers according -to the placement algorithm, where N is the maximum replication factor -supported by the storage system. Because the system is using consistent -hashing, these are the only peers that may share objects with a server +Each Triton server will monitor its N*2 nearest neighbor servers using data +obtained via the SWIM protocol described in failure-detection.txt, where N is the maximum replication factor +supported by the storage system. Because of the placement scheme used by +Triton, these are the only peers that may share objects with a server following a failure. Note that the "nearest" servers in this context refer to servers that are virtually adjacent according to the object placement algorithm. There is no implication of physical locality. @@ -76,6 +76,13 @@ digraph rosd_rec_reb { === Principles and terminology +* Note from rosd-replication.txt that the default mode of replication relays +data from the primary server to its replicas in a chain configuration. The +master relays to replica 1, which then relays to replica 2, and so on. In +this chain configuration we will refer to the replica (or master) +immediately preceding a local object as its _replication parent_, while we +refer to a replica following a local object as its _replication child_. + * Rebuilding servers will _pull_ data from their peers. That is, each server individually determines which objects it is missing after a failure (by querying potential parent servers) and then copies data from @@ -127,9 +134,10 @@ incoming requests from clients or nacks with "try again in N seconds" . query the two adjacent servers in the ID space to get a list of missing object IDs that need to be created locally * those two servers can identify all missing local replicas because they are the -only servers that could be replication parents for local objects +only servers that can immediately precede the local server in a replica +chain -. create a local placeholder (ie, a <<stub,stub>>) for each missing object +. create a local placeholder (i.e., a <<stub,stub>>) for each missing object . scan local objects in order to prepare to answer queries from adjacent servers for missing objects @@ -161,7 +169,7 @@ From a server point of view, writes (and other modifications) will be forwarded to stubs in the same manner as they would be forwarded to a normal object. The updates will be applied to stubs just they would be applied to a normal object. This behavior allows stubs to stay in sync -even while rebuilds are occurring at the same time (ie, avoiding the problem +even while rebuilds are occurring at the same time (i.e., avoiding the problem of "chasing a moving target" when trying to rebuild). The "rebuild" server state is equivalent to the "nominal" server state in that client @@ -218,9 +226,18 @@ the safest general approach will be to simply restart the recovery process on all affected servers. The number of affected servers will be expanded to include those within maximum replication distance of both failed nodes. -The recovery process will be restarted from scratch to make sure that we -don't accidentally assume that we have finished creating stubs for an object -already if it actually requires additional stubs due to the second failure. +The recovery process will be restarted from scratch in this scenario rather +than attempting to account for objects that have already been processed in +response to the previous failure. For example, consider a server that (as +part of recovery) has created stubs for half of the failed objects that it +must reconstruct. If another failure occurs at this point, we cannot make +any assumptions about the integrity of the ID space that the server has +previously processed, because the new failure may have impacted portions of +that same ID range. It is also possible that another immediately adjacent +server has failed. The local server will therefore stop its current +recovery procedure and restart it based on the new state of the system to +insure that all objects are accounted for and that there are no race conditions +between competing recovery procedures. This approach requires that stub creation be idempotent in case a server calculates that it needs to create a new stub when there is one already in place from a @@ -238,7 +255,7 @@ reconstruction. [[rebuild]] The Triton rebuild process is driven by servers that hold secondary objects -that are out of sync with their primaries (ie, stubs). These servers "pull" data +that are out of sync with their primaries (i.e., stubs). These servers "pull" data from primary (or parent) objects. Once they have pulled all necessary data for a given object, then the stub designation will be removed from that object and clients will be allowed to interact with it as usual. @@ -249,14 +266,14 @@ will want to batch these operations together and generate a schedule for an optimal order to perform rebuilds. . for each stub object: -.. retrieve update state (ie, what updates have been applied to the stub +.. retrieve update state (i.e., what updates have been applied to the stub already- see the replication documentation for details) ... may be "0" if it this is a new stub with no data .. contact parent to request rebuild data .. on parent: ... checkpoint the object in question -... stream a _pickled_ (ie, serialized) copy of any object data that is newer than the replica's +... stream a _pickled_ (i.e,, serialized) copy of any object data that is newer than the replica's update state to the replica ... delete checkpoint .. on local server: @@ -301,7 +318,7 @@ At a global level, there will be a subset of servers that each need to pull replica data for some objects from some peers. In the long run a research contribution here would be to determine an optimal global schedule for the data transfers so that we utilize the network and disks effectively -and priorities critical objects. +and prioritize critical objects. We won't do this in the prototype, but here are some thing that we will want to consider in future work: @@ -341,7 +358,7 @@ These operations will probably be implemented inefficiently at first until we implement the first cut of recovery and rebuild to make sure that we have the right API and use cases. Then we can go back and optimize these as needed within the TOSD. The TOSD already utilizes an internal database that -can be leverage for fast range queries and bulk operations. +can be leveraged for fast range queries and bulk operations. == Interacting with servers that are rebuilding @@ -355,13 +372,14 @@ server activities lead to which client responses: * attempt to access a server that is in "recovery state": ** receive a nack saying "try again in N seconds" -* attempted to access an object stub (ie, an object that needs to be rebuilt +* attempted to access an object stub (i.e., an object that needs to be rebuilt or resynced): ** receive a nack saying "try primary instead" * attempt to access a server that is down ** timeout, perceived transient network error, or retryable error -** retryable errors includes server responses that say "I'm dead" +** retryable errors includes server nacks indicating that the server has +failed and is not valid for any storage operations === Possible optimization: allowing client access to rebuilding replicas @@ -380,7 +398,7 @@ as long as they have at least started or queued up the rebuild process for that object. Servers could even initiate a rebuild on demand if a client tried to access a rebuilding replica. -This optimization will be explored in future work. It is not necessary +These optimization will be explored in future work. It is not necessary for prototype correctness. === Explicitly failing a server diff --git a/code/doc/resilience/resilience-book.txt.in b/code/doc/resilience/resilience-book.txt.in index 6708bef..d988970 100644 --- a/code/doc/resilience/resilience-book.txt.in +++ b/code/doc/resilience/resilience-book.txt.in @@ -1,21 +1,19 @@ -Triton server-driven response to storage failures -================================================= +Triton: Data Organization, Replication, and Resilience +====================================================== include::@abs_srcdir@/resiliency-overview.txt[] -include::@abs_srcdir@/failure-detection.txt[] - include::@abs_srcdir@/object-placement.txt[] include::@abs_srcdir@/rosd-replication.txt[] +include::@abs_srcdir@/failure-detection.txt[] + include::@abs_srcdir@/object-rebuild.txt[] include::@abs_srcdir@/client-fault-handling.txt[] include::@abs_srcdir@/non-idempotent-retry.txt[] -include::@abs_srcdir@/fault-injection.txt[] - include::@abs_srcdir@/glossary.txt[] diff --git a/code/doc/resilience/resiliency-overview.txt b/code/doc/resilience/resiliency-overview.txt index c5f2601..35a5bf2 100644 --- a/code/doc/resilience/resiliency-overview.txt +++ b/code/doc/resilience/resiliency-overview.txt @@ -10,7 +10,7 @@ resiliency. This document outlines the Triton object server approach to resiliency. Triton employs a server-driven response to storage failures, in which servers actively detect failures and autonomously repair the storage system in response to those failures. Some of the most -notable design features of Triton from a resiliency perspective include: +notable design features of Triton, from a resiliency perspective, include: * integrated replication mechanisms that can leverage commodity storage hardware @@ -18,7 +18,7 @@ hardware affected objects * scalable algorithms for failure detection, object placement, and replication -* tunable semantics +* tunable consistency semantics Note that the Triton object storage system does not directly provide a file system interface or a traditional POSIX name space. It simply provides access to @@ -49,7 +49,4 @@ respond to various fault scenarios. transient failures, especially in the context of non-idempotent storage operations -* fault-injection.txt: how to inject various faults for -prototyping/testing/validation purposes - * glossary.txt: definitions for common terms that appear in the above documents diff --git a/code/doc/resilience/rosd-replication.txt b/code/doc/resilience/rosd-replication.txt index d9ec7f0..43479a6 100644 --- a/code/doc/resilience/rosd-replication.txt +++ b/code/doc/resilience/rosd-replication.txt @@ -25,35 +25,46 @@ simultaneously to shared objects or unique objects. Aggregate read performance is a high priority for the ASG project. I/O access patterns may or may not be bursty. -We anticipate that HPC workloads often involve highly coordinated clients -and high level I/O libraries. Such workloads do not typically require POSIX -read/write semantics (notably, MPI-IO, NFS, and PVFS have proved successful -in many deployments despite a lack of strict POSIX conformance). The ROSD -will support a range of possible semantics at run time. The semantics will -therefore be tunable per-application to reflect the needs of each -application rather than enforcing POSIX semantics for all applications. +The triton servers provide a distributed object storage service rather +than a conventional file system interface, but we expect that file +system interfaces will be built atop the object storage interface. +The presence of a file system layer atop the system does not imply +that POSIX read/write consistency semantics are required at the object +level, however. HPC workloads often involve highly coordinated clients +and high level I/O libraries. Such workloads do not require POSIX +read/write semantics (notably, MPI-IO, NFS, +and PVFS have proved successful in many deployments despite a lack of +strict POSIX conformance). The ROSD semantics will therefore be tunable +per-application to reflect the needs of each application, regardless of the +application interface that it is using. ==== Hardware An exascale storage system will likely rely on commodity storate components in order to achieve performance targets in a cost-effective manner. In this deployment model, there is no shared storage between -servers. All redundancy must be supplied in software by the +servers. Server-level redundancy must be supplied in software by the storage system. ==== Redundancy model This document describes the replication of data, in which all data is -fully copied to one or more redundant locations in order to tolerate +fully copied to one or more redundant servers in order to tolerate failure. An alternative to this approach is to use RAID or erasure coding -algorithms to tolerate failures while consuming less storage capacity. +algorithms across servers to tolerate failures while consuming less storage capacity. Distributed erasure coding algorithms impose significant challenges to preserving consistency in the presence of concurrent writes, however. Our prototype will therefore focus on object replication in the short -term and explore erasure coding as a longer range option. We believe -that in the future the system is likely to support a model in which replicated +term and explore erasure coding as a longer range option. In the future, the system is likely to support a model in which replicated data is automatically migrated into an erasure coded format in order to -reclaim storage capacity once the data is quiescent. +reclaim storage capacity, once the data is quiescent. + +We also anticipate that the local storage devices attached to servers +will support RAID configurations as well. This functionality will allow +servers to tolerate local disk failures without perturbing other servers. In future work we will use mathematical models in conjunction +with discrete event simulation to determine the appropriate balance +of local redundancy and inter-server redundancy based on the expected +component failure rate and the rebuild time of inter-server replication. ==== Rebuild @@ -68,7 +79,7 @@ sets that are not fully synchronized. More specifically, the system should not block access to objects while they are in the process of rebuilding. The replication protocol should be capable of accepting new write operations while simultaneously rebuilding failed objects. This is similar to the -behavior or traditional RAID disk arrays, in which availability is +behavior of traditional RAID disk arrays, in which availability is maintained even while the array is in a degraded mode. [[sec-overview]] @@ -144,10 +155,10 @@ database entries. * Triton object replication deals only with _replicated_ data, and not _partitioned_ data. We are replicating -at an object granularity, not at striped file granularity. Any semantics +at an object granularity, not at striped file or striped object granularity. Any semantics required across striped sets will be enforced at a higher level of the storage hierarchy. This means that a distributed object storage system need -not implement locking or transactional semantics _across stripe boundaries_. +not implement locking or transactional semantics _across objects_. * Reads that overlap with concurrent writes are allowed to return undefined results. We expect HPC applications to be coordinated at a higher level so @@ -183,7 +194,12 @@ independently. In addition, many distributed Internet service storage systems need only support write-once semantics. For example, it may store key/value pairs or images that will not be updated once written. In contrast, HPC systems must at least support appendable objects in order to -handle the volume of data stored by large scale simulations. +handle the volume of data stored by large scale simulations, which often cannot be +transferred in a single write operation. The Triton +storage system is intended to persistently and resiliently store all +classes of persistent objects. That includes both bulk data (which is often +written once) and metadata (including indices which might be updated +frequently). === Challenges @@ -192,9 +208,15 @@ must address. First of all, it must avoid serializing I/O operations to shared objects. Concurrent read and write operations should be able to proceed immediately in order to maximize bandwidth. Secondly, the replication must preserve consistency across replicas even when multiple -write operations are in flight concurrently. Finally, each replica should -be self-describing, in that the system can query replicas to determine if -they are synchronized, and if not, exactly how they diverge. +write operations are in flight concurrently. In this scenario, +each replica may be applying interleaved or even overlapping write +operations simultaneously without synchronization from other replicas. Even +in this scenario, the storage system must honor well-defined read and write +semantics at all times and preserve replica consistency even if failures +occur while writing. +Finally, each replica should be self-describing, in that the system +can query replicas to determine if they are synchronized, and if not, +exactly how they diverge. == The Triton ROSD replication protocol @@ -300,7 +322,7 @@ increasing numbers rather than time stamps, and write ordering will be resolved before reads occur. The default rules for replica divergence in the ROSD protocol will be defined as follows: Triton object replicas can -diverge as long as concurrent client I/O operations are in flight. Once a +diverge as long as concurrent client write operations are in flight. Once a write completes from a client's perspective, it is guaranteed to be visible at all replicas and in the correct order relative to any _subsequent_ reads or writes. @@ -362,13 +384,15 @@ approach. The first is that the server is unaware of the scope of the overall write operation and therefore cannot schedule accordingly. It only sees a small part of the overall operation at a time. The second drawback is that it makes it difficult to preserve atomicity (and POSIX semantics) -across a write operation that has been broken into distinct segments. +across a write operation that has been broken into distinct segments, when +this is required. The approach taken by Triton instead is to establish a "pipeline" operation, in which the client informs the server of the entire desired transfer, and the server then chooses its own plan for dividing the transfer into smaller components. The server is aware of the entire update and can schedule or -overlap operations in an optimal manner. +overlap operations in an optimal manner, similar to previous reseach in +server-directed I/O <<Seamons>>. In order to implement this, the TOSD local storage abstraction provides a transaction construct that allows update numbers to refer to an arbitrary number of write @@ -394,7 +418,7 @@ alternative model would be to accept the write operation at the master as usual, but then have the master fan out updates to a set of replicas simultaneously. This approach may be more appropriate for small, latency bound operations, while the chain approach is likely more appropriate for -large bandwidth-bound operations. We evaluate the fan-out replication +large bandwidth-bound operations. We will evaluate the fan-out replication model and develop heuristics to automatically select the appropriate protocol for each type of operation. @@ -434,7 +458,7 @@ other things. === Discussion -A ROSD server has to handle four failure classes: +The ROSD protocol has to handle four failure classes: . Network failure while communicating with a remote replica * Retry implicitly via RPC unless/until the Traffic Cop (which in turn @@ -499,3 +523,7 @@ WA, November 2006. - [[[Thomas]]] Robert H. Thomas. "A majority consensus approach to concurrency control for multiple copy databases". ACM Transactions on Database Systems 4 (2): 180–209. +- [[[Searmons]]] Seamons, K. E., Chen, Y., Jones, P., Jozwiak, J., and Winslett, M. 1995. + Server-directed collective I/O in Panda. In Proceedings of Supercomputing ’95. IEEE + Computer Society + Press, Los Alamitos, Calif. hooks/post-receive -- Triton Repository

1 0

Triton Repository branch, master, updated. 69395960a5db99a0607dbc4e88ca97d868848490
by noreply＠mcs.anl.gov 28 Feb '12

28 Feb '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton Repository". The branch, master has been updated via 69395960a5db99a0607dbc4e88ca97d868848490 (commit) from e6a02207d2b82af9bd16d1b9e3649f28af8ae573 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 69395960a5db99a0607dbc4e88ca97d868848490 Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Tue Feb 28 02:53:34 2012 -0600 Add STONITH to glossary.txt ----------------------------------------------------------------------- Summary of changes: code/doc/resilience/glossary.txt | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) Diff of changes: diff --git a/code/doc/resilience/glossary.txt b/code/doc/resilience/glossary.txt index 3c35ec7..e43e9b4 100644 --- a/code/doc/resilience/glossary.txt +++ b/code/doc/resilience/glossary.txt @@ -31,4 +31,7 @@ ID onto a set of server IDs. This is a deterministic algorithm. The current implementation uses consistent hashing to map an object ID to the N closest server IDs in a ring. This is similar to PAST, except that all servers (and their IDs) are known ahead of time from gossip. - +* STONITH: Acronym for Shoot The Other Node In The Head, popularized by the +Linux-HA Heartbeat software package. STONITH is a mechanism where a node +can physically shutdown or power off another node to fence it. This is +typically done via IPMI power control. hooks/post-receive -- Triton Repository

1 0

Triton Repository branch, master, updated. e6a02207d2b82af9bd16d1b9e3649f28af8ae573
by noreply＠mcs.anl.gov 28 Feb '12

28 Feb '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton Repository". The branch, master has been updated via e6a02207d2b82af9bd16d1b9e3649f28af8ae573 (commit) from 3787341e422c6d143cdb6c770569f3c7f1d7714d (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit e6a02207d2b82af9bd16d1b9e3649f28af8ae573 Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Tue Feb 28 02:47:56 2012 -0600 Minor edits to non-idempotent-retry.txt for spelling. ----------------------------------------------------------------------- Summary of changes: code/doc/resilience/non-idempotent-retry.txt | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) Diff of changes: diff --git a/code/doc/resilience/non-idempotent-retry.txt b/code/doc/resilience/non-idempotent-retry.txt index edf09fa..db3264a 100644 --- a/code/doc/resilience/non-idempotent-retry.txt +++ b/code/doc/resilience/non-idempotent-retry.txt @@ -134,7 +134,7 @@ client is actually a proxy for multiple independent processes. This scenario may arise when multiple cores or processes on a node share a single client instance or if I/O function calls are forwarded from lightweight compute node kernels to intermediate gateway nodes. As an example of the latter -case, the current generate if IBM BlueGene/P systems use I/O forwarding +case, the current generation IBM BlueGene/P systems use I/O forwarding nodes that each service I/O operations from many compute nodes and compute cores through a single file system client interface. @@ -173,7 +173,7 @@ of the object when the operation was first performed. === Message transport optimizations Both the Ceph and Lustre retry mechanisms include an additional optimization -for use network transports that prevent duplicate delivery (such as TCP/IP). +for use in network transports that prevent duplicate delivery (such as TCP/IP). In that case, each request includes a retry counter or retry flag to differentiate requests that have been explicitly retried by a client vs. those that have not. If the system trusts the network to prevent delivery hooks/post-receive -- Triton Repository

1 0

Triton Repository branch, master, updated. 3787341e422c6d143cdb6c770569f3c7f1d7714d
by noreply＠mcs.anl.gov 28 Feb '12

28 Feb '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton Repository". The branch, master has been updated via 3787341e422c6d143cdb6c770569f3c7f1d7714d (commit) from cf4cfbc4dda0fa197c4f5cc18309dcaf1585aaa5 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 3787341e422c6d143cdb6c770569f3c7f1d7714d Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Tue Feb 28 02:31:39 2012 -0600 Minor edits to client-fault-handling.txt for spelling ----------------------------------------------------------------------- Summary of changes: code/doc/resilience/client-fault-handling.txt | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) Diff of changes: diff --git a/code/doc/resilience/client-fault-handling.txt b/code/doc/resilience/client-fault-handling.txt index 4a32c43..da22561 100644 --- a/code/doc/resilience/client-fault-handling.txt +++ b/code/doc/resilience/client-fault-handling.txt @@ -46,7 +46,7 @@ times for requests over time and use this information to calculate appropriate timeout values for future operations. The actual client communication logic will simply query the traffic cop on each operation to determine how long it should wait for a response. If a response is not -received, then the traffic is consulted to determine whether the client +received, then the traffic cop is consulted to determine whether the client should retry to the same server, retry to a different server, or refresh its system state information. The traffic cop is a centralized location for local policy decisions and may use information from a variety hooks/post-receive -- Triton Repository

1 0

Triton Repository branch, master, updated. cf4cfbc4dda0fa197c4f5cc18309dcaf1585aaa5
by noreply＠mcs.anl.gov 28 Feb '12

28 Feb '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton Repository". The branch, master has been updated via cf4cfbc4dda0fa197c4f5cc18309dcaf1585aaa5 (commit) from e82053ae2b81c6cf0256e3432fef7883c6494dbb (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit cf4cfbc4dda0fa197c4f5cc18309dcaf1585aaa5 Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Tue Feb 28 02:23:18 2012 -0600 Minor edits to object-rebuild.txt for spelling ----------------------------------------------------------------------- Summary of changes: code/doc/resilience/object-rebuild.txt | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) Diff of changes: diff --git a/code/doc/resilience/object-rebuild.txt b/code/doc/resilience/object-rebuild.txt index a59a79d..649537d 100644 --- a/code/doc/resilience/object-rebuild.txt +++ b/code/doc/resilience/object-rebuild.txt @@ -194,7 +194,7 @@ nack telling them to use the primary instead ==== Rationale for disabling client access during recovery Clients request will either be queued or retried until recovery is -complete. This be behavior will be enforced (at least in the prototype +complete. This behavior will be enforced (at least in the prototype implementation) in order to insure correct semantics. The object placement will be unresolvably inconsistent until recovery is complete. For example, if a client attempts to access a replica that doesn't exist, @@ -380,7 +380,7 @@ as long as they have at least started or queued up the rebuild process for that object. Servers could even initiate a rebuild on demand if a client tried to access a rebuilding replica. -This optimizations will be explored in future work. They are not necessary +This optimization will be explored in future work. It is not necessary for prototype correctness. === Explicitly failing a server hooks/post-receive -- Triton Repository

1 0

Triton Repository branch, master, updated. e82053ae2b81c6cf0256e3432fef7883c6494dbb
by noreply＠mcs.anl.gov 28 Feb '12

28 Feb '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton Repository". The branch, master has been updated via e82053ae2b81c6cf0256e3432fef7883c6494dbb (commit) from ec7e93775ebb03e7def31bb24b6a57f5e99b0cbe (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit e82053ae2b81c6cf0256e3432fef7883c6494dbb Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Tue Feb 28 01:58:37 2012 -0600 Minor edits to rosd-replication.txt for spelling and sentence structure. ----------------------------------------------------------------------- Summary of changes: code/doc/resilience/rosd-replication.txt | 22 +++++++++++----------- 1 files changed, 11 insertions(+), 11 deletions(-) Diff of changes: diff --git a/code/doc/resilience/rosd-replication.txt b/code/doc/resilience/rosd-replication.txt index 7145dfe..d9ec7f0 100644 --- a/code/doc/resilience/rosd-replication.txt +++ b/code/doc/resilience/rosd-replication.txt @@ -51,7 +51,7 @@ Distributed erasure coding algorithms impose significant challenges to preserving consistency in the presence of concurrent writes, however. Our prototype will therefore focus on object replication in the short term and explore erasure coding as a longer range option. We believe -that the system is likely to eventually support a model in which replicated +that in the future the system is likely to support a model in which replicated data is automatically migrated into an erasure coded format in order to reclaim storage capacity once the data is quiescent. @@ -116,18 +116,18 @@ important general contribution is a more formal approach to defining the semantics of database transactions. For example, database transactions are often described in terms of ACID properties: -* Atomicity: Either all of the modifications for a transaction are performed +* *Atomicity*: Either all of the modifications for a transaction are performed or none of them are. -* Consistency: Each successful transaction only commits legal results. -* Isolation: Other transactions cannot see data in an intermediate state from +* *Consistency*: Each successful transaction only commits legal results. +* *Isolation*: Other transactions cannot see data in an intermediate state from another transaction. -* Durability: once a user has been notified of success, the transaction will +* *Durability*: once a user has been notified of success, the transaction will persist and not be undone. Databases typically implement these semantics using protocols such as two phase locking <<Bernstein>>. Two phase locking and its many derivatives (such as presumed commit, presumed abort, etc) -are limited in terms of concurrent write performance, however, due to the fact +are limited in terms of concurrent write performance due to the fact that they either impose synchronization across nodes or impose ordering dependencies across nodes. In general, distributed database replication protocols are relatively costly because they are trying to solve a fundamentally more difficult @@ -161,7 +161,7 @@ Current parallel file system replication protocols <<Schmuck-GPFS>>,<<Welch>>,<<Weil-Ceph>>,<<Abd-El-Malek>> fail to meet our exascale storage requirements in one or more of the following ways: -* Poor performance for concurrent writes, brought about either by distributed +* Poor performance for concurrent writes, brought about by distributed locking, serialized writes, or multi-step commit protocols. * Lack of ability to explicitly identify the state of objects, which complicates re-synchronization after repairs and also makes it difficult to @@ -242,7 +242,7 @@ actually applied to storage, the final result visible to readers will be as if they were applied serially according to their update number. ** For example, if update number 35 arrives at an object and finds that the affected region already contains update 36, then update 35 is discarded. -Thsi is similar to the Thomas Write Rule semantics often used in database +This is similar to the Thomas Write Rule semantics often used in database systems <<Thomas>>. The update numbers are _not_ visible to clients. They are constructed @@ -340,7 +340,7 @@ software (not the application itself, but rather a kernel driver or storage system library) will automatically retry failed storage operations. If a write fails with a recoverable error, then the client library will simply retry the write operation either to the same server or to a new server if the original -master as failed. +master has failed. === Variations and optimizations @@ -377,7 +377,7 @@ local transaction at the beginning of a pipeline and close the transaction once the pipeline is complete. An important thing to note, however, is that the pipeline can fail, stall, -or timeout at any time with affecting any other I/O operations on a server, +or timeout at any time without affecting any other I/O operations on a server, whether those I/O operations apply to the same object or a different object. Thus there is no risk of deadlocking the system if a pipeline fails. A client can restart a pipeline at any point and the ROSD will eventually time @@ -434,7 +434,7 @@ other things. === Discussion -An ROSD server has to handle four failure classes: +A ROSD server has to handle four failure classes: . Network failure while communicating with a remote replica * Retry implicitly via RPC unless/until the Traffic Cop (which in turn hooks/post-receive -- Triton Repository

1 0

Triton Repository branch, master, updated. ec7e93775ebb03e7def31bb24b6a57f5e99b0cbe
by noreply＠mcs.anl.gov 28 Feb '12

28 Feb '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton Repository". The branch, master has been updated via ec7e93775ebb03e7def31bb24b6a57f5e99b0cbe (commit) from ccd721cc5edd113fa0f371602eed43f3177c10bf (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit ec7e93775ebb03e7def31bb24b6a57f5e99b0cbe Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Tue Feb 28 01:13:13 2012 -0600 Minor edits to failure-detection.txt for spelling, sentence structure ----------------------------------------------------------------------- Summary of changes: code/doc/resilience/failure-detection.txt | 18 +++++++++--------- 1 files changed, 9 insertions(+), 9 deletions(-) Diff of changes: diff --git a/code/doc/resilience/failure-detection.txt b/code/doc/resilience/failure-detection.txt index 9fbf9d3..9763a12 100644 --- a/code/doc/resilience/failure-detection.txt +++ b/code/doc/resilience/failure-detection.txt @@ -49,7 +49,7 @@ occur within a bounded amount of time. === Failure Detection Background and Terminology -Failure detectors general consist of two parts: +Failure detectors generally consist of two parts: * Failure detection *mechanism*: the mechanism used by a single node to identify the failure of another @@ -198,19 +198,19 @@ can take action when a specific peer server fails. We must handle a number of different failure conditions: -* Partial failures: we cannot assume fail-stop conditions in all cases. +* *Partial failures*: we cannot assume fail-stop conditions in all cases. In scenarios where a server has experienced a storage failure or a software -failure then the server itself (or its peers) may explicitly force a failure +failure, the server itself (or its peers) may explicitly force a failure mode. In this case the server is still capable of communication but will be treated as a fully failed node for replication and rebuild purposes. -* Client connectivity: even though a server has lost connectivity with its +* *Client connectivity*: even though a server has lost connectivity with its peer servers, it may still be capable of communicating with clients. Therefore we should expect that a "failed" server could receive client requests at any time. It must handle this condition gracefully and inform -that client of the current state so that it can respond accordingly. +clients of the current state so that they can respond accordingly. -* Network partitions: Network partitions may occur, and we can't differentiate between a +* *Network partitions*: Network partitions may occur, and we can't differentiate between a network partition and node failure. For this reason, we treat all node failures as network partitions. A single node failure is modeled as a network partition where only one node is partitioned from all others. @@ -219,7 +219,7 @@ partitions occur. If a server detects that it is in a partition that is in the minority of the total collection of servers, then it will automatically enter a failure mode. -* Server state vs. replication groups: If a server enters a failure mode, +* *Server state vs. replication groups*: If a server enters a failure mode, then it will be considered to be ineligible for storage of all objects. There is no steady-state server failure mode in which it handles some objects normally but fails for other objects. @@ -294,9 +294,9 @@ failure mode within a bounded amount of time based on the approximate size of the system and the gossip messaging rate. Once a server enters a failure state, it will no longer accept read or write -requests (or any object request) from clients. If possible, it will return a +requests (or any object request) from clients. If possible, the server will return a negative acknowledgment with a well-known error code so that the client -gets immediate feedback that it should fail over to a different server or +gets immediate feedback in order to fail over to a different server or else request updated system state information from another server. ==== Rebuilding: hooks/post-receive -- Triton Repository

1 0

Triton-private Repository branch, master, updated. ae59435a6f4aaf4f56a87cfa91b7fe96c4eb951f
by noreply＠mcs.anl.gov 25 Feb '12

25 Feb '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton-private Repository". The branch, master has been updated via ae59435a6f4aaf4f56a87cfa91b7fe96c4eb951f (commit) from 80382e6f15e1c19a20de782d336b952bd101f70d (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit ae59435a6f4aaf4f56a87cfa91b7fe96c4eb951f Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Sat Feb 25 22:30:25 2012 -0600 Update graph creation so that rotated text works in png images. This may only work on ubuntu. Also make thread names match in time and latency graphs ----------------------------------------------------------------------- Summary of changes: papers/2011/aesop/fig/Makefile | 2 +- papers/2011/aesop/fig/gendata.sh | 29 ++++++++++++++++++++++++----- 2 files changed, 25 insertions(+), 6 deletions(-) Diff of changes: diff --git a/papers/2011/aesop/fig/Makefile b/papers/2011/aesop/fig/Makefile index dea2ee3..dbb0f76 100644 --- a/papers/2011/aesop/fig/Makefile +++ b/papers/2011/aesop/fig/Makefile @@ -28,7 +28,7 @@ clean:: inkscape -d $(DPI) -e $@ $< %.png: %.gnuplot %.data - @echo 'set terminal "png" transparent large; set output "$@"' | gnuplot - $< + @echo 'set terminal "png" transparent large font "/usr/share/fonts/truetype/ttf-liberation/LiberationSans-Regular.ttf, 12"; set output "$@"' | gnuplot - $< %.pdf: %.gnuplot %.data diff --git a/papers/2011/aesop/fig/gendata.sh b/papers/2011/aesop/fig/gendata.sh index c89c375..7a175e7 100755 --- a/papers/2011/aesop/fig/gendata.sh +++ b/papers/2011/aesop/fig/gendata.sh @@ -67,12 +67,21 @@ then for type in aesop thread thread-nb thread-per-op thread-pool event; do j=0; - #for nodes in 16 32 64; for nodes in 64; do procs=$((nodes*16)) - #echo -n "$type-$procs " >> ${test}.data; - echo -n "$type " >> ${test}.data; + + if [ $type = "thread" ]; + then + prettytype="thread-per-client" + elif [ $type = "thread-nb" ]; + then + prettytype="thread-per-client-nb" + else + prettytype=$type + fi + + echo -n "$prettytype " >> ${test}.data; val=$(cat ${basepath}/${base}/${type}/client-${type//-/}-${base}-${nodes}-* 2>/dev/null | grep "max=" | perl -ne 'if (/min=(\S+) max=(\S+) median=(\S+) first=(\S+) third=(\S+) mean=(\S+) stddev=(\S+)/) { $min=$1; $max=$2; $median=$3; $first=$4; $third=$5; $mean=$6; $stddv=$7; $iter[$i] = { min => $min, max => $max, med => $median, fst => $first, trd => $third }; $i += 1;} BEGIN { $i = 0; @iter = (); } END { my @srt = sort { $b->{max} <=> $a->{max} } @iter; print "$srt[2]->{fst} $srt[2]->{min} $srt[2]->{max} $srt[2]->{trd}\n"; }' ) if [ -z "${val}" ]; then @@ -95,8 +104,18 @@ then for nodes in 64; do procs=$((nodes*16)) - #echo -n "$type-$procs " >> ${test}.data; - echo -n "$type " >> ${test}.data; + + if [ $type = "thread" ]; + then + prettytype="thread-per-client" + elif [ $type = "thread-nb" ]; + then + prettytype="thread-per-client-nb" + else + prettytype=$type + fi + + echo -n "$prettytype " >> ${test}.data; val=$(cat ${basepath}/${base}/${type}/client-${type//-/}-${base}-${nodes}-* 2>/dev/null | perl -ne 'if (/min:(\S+) max:(\S+)/) { $min=$1; $max=$2; $iter[$i] = { min => $min, max => $max }; $i += 1;} BEGIN { $i = 0; @iter = (); } END { my @srt = sort { $b->{max} <=> $a->{max} } @iter; print "$srt[2]->{min} $srt[2]->{max}\n"; }') if [ -z "${val}" ]; then val="-"; hooks/post-receive -- Triton-private Repository

1 0

Triton-private Repository branch, master, updated. 80382e6f15e1c19a20de782d336b952bd101f70d
by noreply＠mcs.anl.gov 24 Feb '12

24 Feb '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton-private Repository". The branch, master has been updated via 80382e6f15e1c19a20de782d336b952bd101f70d (commit) via 5916f98dc40ca66a7a8868c3690283b712e7b760 (commit) from 0ac2cdd54bf8715b3a4df4a24acf411b20d52212 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 80382e6f15e1c19a20de782d336b952bd101f70d Merge: 5916f98dc40ca66a7a8868c3690283b712e7b760 0ac2cdd54bf8715b3a4df4a24acf411b20d52212 Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Fri Feb 24 17:11:12 2012 -0600 Merge branch 'master' of git.mcs.anl.gov:triton-private commit 5916f98dc40ca66a7a8868c3690283b712e7b760 Author: Kevin Harms <harms(a)alcf.anl.gov> Date: Fri Feb 24 17:11:03 2012 -0600 Add latency and total time graphs ----------------------------------------------------------------------- Summary of changes: papers/2011/aesop/fig/Makefile | 5 +- papers/2011/aesop/fig/gendata.sh | 49 +++++++++++++++++++- .../fig/{read-hist.gnuplot => read-lat.gnuplot} | 15 +++--- .../{read-hist.gnuplot => read-null-lat.gnuplot} | 15 +++--- .../{read-hist.gnuplot => read-null-time.gnuplot} | 15 +++--- .../fig/{read-hist.gnuplot => read-time.gnuplot} | 15 +++--- .../fig/{read-hist.gnuplot => write-lat.gnuplot} | 15 +++--- .../{read-hist.gnuplot => write-null-lat.gnuplot} | 15 +++--- .../{read-hist.gnuplot => write-null-time.gnuplot} | 15 +++--- .../fig/{read-hist.gnuplot => write-time.gnuplot} | 15 +++--- 10 files changed, 115 insertions(+), 59 deletions(-) copy papers/2011/aesop/fig/{read-hist.gnuplot => read-lat.gnuplot} (91%) copy papers/2011/aesop/fig/{read-hist.gnuplot => read-null-lat.gnuplot} (91%) copy papers/2011/aesop/fig/{read-hist.gnuplot => read-null-time.gnuplot} (91%) copy papers/2011/aesop/fig/{read-hist.gnuplot => read-time.gnuplot} (91%) copy papers/2011/aesop/fig/{read-hist.gnuplot => write-lat.gnuplot} (91%) copy papers/2011/aesop/fig/{read-hist.gnuplot => write-null-lat.gnuplot} (91%) copy papers/2011/aesop/fig/{read-hist.gnuplot => write-null-time.gnuplot} (91%) copy papers/2011/aesop/fig/{read-hist.gnuplot => write-time.gnuplot} (91%) Diff of changes: diff --git a/papers/2011/aesop/fig/Makefile b/papers/2011/aesop/fig/Makefile index 0c5731c..dea2ee3 100644 --- a/papers/2011/aesop/fig/Makefile +++ b/papers/2011/aesop/fig/Makefile @@ -8,8 +8,9 @@ $(warning $(dest)) GRAPHS=read.pdf write.pdf read-null.pdf write-null.pdf \ read-hist.pdf write-hist.pdf read-null-hist.pdf write-null-hist.pdf \ - read-mem.pdf write-mem.pdf read-null-mem.pdf write-null-mem.pdf - + read-mem.pdf write-mem.pdf read-null-mem.pdf write-null-mem.pdf \ + read-lat.pdf write-lat.pdf read-null-lat.pdf write-null-lat.pdf \ + read-time.pdf write-time.pdf read-null-time.pdf write-null-time.pdf all:: $(dest) $(GRAPHS) diff --git a/papers/2011/aesop/fig/gendata.sh b/papers/2011/aesop/fig/gendata.sh index 1dacada..c89c375 100755 --- a/papers/2011/aesop/fig/gendata.sh +++ b/papers/2011/aesop/fig/gendata.sh @@ -7,7 +7,7 @@ shopt -s extglob basepath=../data/fusion test=$1 -base=${1%-@(hist|mem)} +base=${1%-@(hist|mem|lat|time)} file=$1; if [ $test = $base -o $test = "${base}-hist" ]; @@ -60,6 +60,53 @@ then echo "" >> ${test}.data done +elif [ $test = "${base}-lat" ]; +then + echo "#type first min max third x" > ${test}.data; + i=1; + for type in aesop thread thread-nb thread-per-op thread-pool event; + do + j=0; + #for nodes in 16 32 64; + for nodes in 64; + do + procs=$((nodes*16)) + #echo -n "$type-$procs " >> ${test}.data; + echo -n "$type " >> ${test}.data; + val=$(cat ${basepath}/${base}/${type}/client-${type//-/}-${base}-${nodes}-* 2>/dev/null | grep "max=" | perl -ne 'if (/min=(\S+) max=(\S+) median=(\S+) first=(\S+) third=(\S+) mean=(\S+) stddev=(\S+)/) { $min=$1; $max=$2; $median=$3; $first=$4; $third=$5; $mean=$6; $stddv=$7; $iter[$i] = { min => $min, max => $max, med => $median, fst => $first, trd => $third }; $i += 1;} BEGIN { $i = 0; @iter = (); } END { my @srt = sort { $b->{max} <=> $a->{max} } @iter; print "$srt[2]->{fst} $srt[2]->{min} $srt[2]->{max} $srt[2]->{trd}\n"; }' +) + if [ -z "${val}" ]; then + val="-"; + fi + echo -n "${val} " >> ${test}.data; + echo "$i.$j" >> ${test}.data; + j=$((j+3)) + done + i=$((i+1)); + done +elif [ $test = "${base}-time" ]; +then + echo "#type min max x" > ${test}.data; + i=1; + for type in aesop thread thread-nb thread-per-op thread-pool event; + do + j=0; + #for nodes in 16 32 64; + for nodes in 64; + do + procs=$((nodes*16)) + #echo -n "$type-$procs " >> ${test}.data; + echo -n "$type " >> ${test}.data; + val=$(cat ${basepath}/${base}/${type}/client-${type//-/}-${base}-${nodes}-* 2>/dev/null | perl -ne 'if (/min:(\S+) max:(\S+)/) { $min=$1; $max=$2; $iter[$i] = { min => $min, max => $max }; $i += 1;} BEGIN { $i = 0; @iter = (); } END { my @srt = sort { $b->{max} <=> $a->{max} } @iter; print "$srt[2]->{min} $srt[2]->{max}\n"; }') + if [ -z "${val}" ]; then + val="-"; + fi + echo -n "${val} " >> ${test}.data; + echo "$i.$j" >> ${test}.data; + j=$((j+3)) + done + i=$((i+1)); + done else echo "unknown data type" diff --git a/papers/2011/aesop/fig/read-hist.gnuplot b/papers/2011/aesop/fig/read-lat.gnuplot similarity index 91% copy from papers/2011/aesop/fig/read-hist.gnuplot copy to papers/2011/aesop/fig/read-lat.gnuplot index e3ca292..5711ca2 100644 --- a/papers/2011/aesop/fig/read-hist.gnuplot +++ b/papers/2011/aesop/fig/read-lat.gnuplot @@ -51,12 +51,13 @@ set key title "" set key inside left top vertical Right noreverse enhanced autotitles box linetype -1 linewidth 1.000 set key noinvert samplen 4 spacing 1 width 0 height 0 set key maxcolumns 0 maxrows 0 +unset key unset label unset arrow set style increment default unset style line unset style arrow -set style histogram clustered gap 2 title offset character 0, 0, 0 +#set style histogram clustered gap 2 title offset character 0, 0, 0 unset logscale set logscale y 10 set offsets 0, 0, 0, 0 @@ -95,7 +96,7 @@ set mztics default set mx2tics default set my2tics default set mcbtics default -set xtics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 +set xtics border in scale 1,0.5 mirror rotate by -45 offset character 0, 0, 0 set xtics norangelimit set xtics () set ytics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 @@ -106,7 +107,7 @@ set nox2tics set noy2tics set cbtics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 set cbtics autofreq norangelimit -set title "" +set title "" set title offset character 0, 0, 0 font "" norotate set timestamp bottom set timestamp "" @@ -115,13 +116,13 @@ set rrange [ * : * ] noreverse nowriteback # (currently [8.98847e+307:-8.98847e set trange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) set urange [ * : * ] noreverse nowriteback # (currently [-10.0000:10.0000] ) set vrange [ * : * ] noreverse nowriteback # (currently [-10.0000:10.0000] ) -set xlabel "clients" +set xlabel "Server Type" set xlabel offset character 0, 0, 0 font "" textcolor lt -1 norotate set x2label "" set x2label offset character 0, 0, 0 font "" textcolor lt -1 norotate -set xrange [ * : * ] noreverse nowriteback # (currently [-1.00000:5.00000] ) +set xrange [ 0.5 : 7 ] noreverse nowriteback # (currently [-1.00000:5.00000] ) set x2range [ * : * ] noreverse nowriteback # (currently [-1.00000:5.00000] ) -set ylabel "time(s)" +set ylabel "Time (seconds)" set ylabel offset character 0, 0, 0 font "" textcolor lt -1 rotate by -270 set y2label "" set y2label offset character 0, 0, 0 font "" textcolor lt -1 rotate by -270 @@ -150,5 +151,5 @@ set loadpath set fontpath set fit noerrorvariables GNUTERM = "x11" -plot 'read-hist.data' using 2:xtic(1) title "aesop", '' u 3 ti "thread-per-client", '' u 4 ti "thread-per-client-nb", '' u 5 ti "thread-per-op", '' u 6 ti "thread-pool", '' u 7 ti "event" +plot 'read-lat.data' using 6:2:3:4:5:xticlabels(1) notitle with candlesticks whiskerbars # EOF diff --git a/papers/2011/aesop/fig/read-hist.gnuplot b/papers/2011/aesop/fig/read-null-lat.gnuplot similarity index 91% copy from papers/2011/aesop/fig/read-hist.gnuplot copy to papers/2011/aesop/fig/read-null-lat.gnuplot index e3ca292..19ea914 100644 --- a/papers/2011/aesop/fig/read-hist.gnuplot +++ b/papers/2011/aesop/fig/read-null-lat.gnuplot @@ -51,12 +51,13 @@ set key title "" set key inside left top vertical Right noreverse enhanced autotitles box linetype -1 linewidth 1.000 set key noinvert samplen 4 spacing 1 width 0 height 0 set key maxcolumns 0 maxrows 0 +unset key unset label unset arrow set style increment default unset style line unset style arrow -set style histogram clustered gap 2 title offset character 0, 0, 0 +#set style histogram clustered gap 2 title offset character 0, 0, 0 unset logscale set logscale y 10 set offsets 0, 0, 0, 0 @@ -95,7 +96,7 @@ set mztics default set mx2tics default set my2tics default set mcbtics default -set xtics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 +set xtics border in scale 1,0.5 mirror rotate by -45 offset character 0, 0, 0 set xtics norangelimit set xtics () set ytics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 @@ -106,7 +107,7 @@ set nox2tics set noy2tics set cbtics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 set cbtics autofreq norangelimit -set title "" +set title "" set title offset character 0, 0, 0 font "" norotate set timestamp bottom set timestamp "" @@ -115,13 +116,13 @@ set rrange [ * : * ] noreverse nowriteback # (currently [8.98847e+307:-8.98847e set trange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) set urange [ * : * ] noreverse nowriteback # (currently [-10.0000:10.0000] ) set vrange [ * : * ] noreverse nowriteback # (currently [-10.0000:10.0000] ) -set xlabel "clients" +set xlabel "Server Type" set xlabel offset character 0, 0, 0 font "" textcolor lt -1 norotate set x2label "" set x2label offset character 0, 0, 0 font "" textcolor lt -1 norotate -set xrange [ * : * ] noreverse nowriteback # (currently [-1.00000:5.00000] ) +set xrange [ 0.5 : 7 ] noreverse nowriteback # (currently [-1.00000:5.00000] ) set x2range [ * : * ] noreverse nowriteback # (currently [-1.00000:5.00000] ) -set ylabel "time(s)" +set ylabel "Time (seconds)" set ylabel offset character 0, 0, 0 font "" textcolor lt -1 rotate by -270 set y2label "" set y2label offset character 0, 0, 0 font "" textcolor lt -1 rotate by -270 @@ -150,5 +151,5 @@ set loadpath set fontpath set fit noerrorvariables GNUTERM = "x11" -plot 'read-hist.data' using 2:xtic(1) title "aesop", '' u 3 ti "thread-per-client", '' u 4 ti "thread-per-client-nb", '' u 5 ti "thread-per-op", '' u 6 ti "thread-pool", '' u 7 ti "event" +plot 'read-null-lat.data' using 6:2:3:4:5:xticlabels(1) notitle with candlesticks whiskerbars # EOF diff --git a/papers/2011/aesop/fig/read-hist.gnuplot b/papers/2011/aesop/fig/read-null-time.gnuplot similarity index 91% copy from papers/2011/aesop/fig/read-hist.gnuplot copy to papers/2011/aesop/fig/read-null-time.gnuplot index e3ca292..0ba01be 100644 --- a/papers/2011/aesop/fig/read-hist.gnuplot +++ b/papers/2011/aesop/fig/read-null-time.gnuplot @@ -51,12 +51,13 @@ set key title "" set key inside left top vertical Right noreverse enhanced autotitles box linetype -1 linewidth 1.000 set key noinvert samplen 4 spacing 1 width 0 height 0 set key maxcolumns 0 maxrows 0 +unset key unset label unset arrow set style increment default unset style line unset style arrow -set style histogram clustered gap 2 title offset character 0, 0, 0 +#set style histogram clustered gap 2 title offset character 0, 0, 0 unset logscale set logscale y 10 set offsets 0, 0, 0, 0 @@ -95,7 +96,7 @@ set mztics default set mx2tics default set my2tics default set mcbtics default -set xtics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 +set xtics border in scale 1,0.5 mirror rotate by -45 offset character 0, 0, 0 set xtics norangelimit set xtics () set ytics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 @@ -106,7 +107,7 @@ set nox2tics set noy2tics set cbtics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 set cbtics autofreq norangelimit -set title "" +set title "" set title offset character 0, 0, 0 font "" norotate set timestamp bottom set timestamp "" @@ -115,13 +116,13 @@ set rrange [ * : * ] noreverse nowriteback # (currently [8.98847e+307:-8.98847e set trange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) set urange [ * : * ] noreverse nowriteback # (currently [-10.0000:10.0000] ) set vrange [ * : * ] noreverse nowriteback # (currently [-10.0000:10.0000] ) -set xlabel "clients" +set xlabel "Server Type" set xlabel offset character 0, 0, 0 font "" textcolor lt -1 norotate set x2label "" set x2label offset character 0, 0, 0 font "" textcolor lt -1 norotate -set xrange [ * : * ] noreverse nowriteback # (currently [-1.00000:5.00000] ) +set xrange [ 0.5 : 7 ] noreverse nowriteback # (currently [-1.00000:5.00000] ) set x2range [ * : * ] noreverse nowriteback # (currently [-1.00000:5.00000] ) -set ylabel "time(s)" +set ylabel "Time (seconds)" set ylabel offset character 0, 0, 0 font "" textcolor lt -1 rotate by -270 set y2label "" set y2label offset character 0, 0, 0 font "" textcolor lt -1 rotate by -270 @@ -150,5 +151,5 @@ set loadpath set fontpath set fit noerrorvariables GNUTERM = "x11" -plot 'read-hist.data' using 2:xtic(1) title "aesop", '' u 3 ti "thread-per-client", '' u 4 ti "thread-per-client-nb", '' u 5 ti "thread-per-op", '' u 6 ti "thread-pool", '' u 7 ti "event" +plot 'read-null-time.data' using 4:2:2:3:3:xticlabels(1) notitle with candlesticks # EOF diff --git a/papers/2011/aesop/fig/read-hist.gnuplot b/papers/2011/aesop/fig/read-time.gnuplot similarity index 91% copy from papers/2011/aesop/fig/read-hist.gnuplot copy to papers/2011/aesop/fig/read-time.gnuplot index e3ca292..c1905a2 100644 --- a/papers/2011/aesop/fig/read-hist.gnuplot +++ b/papers/2011/aesop/fig/read-time.gnuplot @@ -51,12 +51,13 @@ set key title "" set key inside left top vertical Right noreverse enhanced autotitles box linetype -1 linewidth 1.000 set key noinvert samplen 4 spacing 1 width 0 height 0 set key maxcolumns 0 maxrows 0 +unset key unset label unset arrow set style increment default unset style line unset style arrow -set style histogram clustered gap 2 title offset character 0, 0, 0 +#set style histogram clustered gap 2 title offset character 0, 0, 0 unset logscale set logscale y 10 set offsets 0, 0, 0, 0 @@ -95,7 +96,7 @@ set mztics default set mx2tics default set my2tics default set mcbtics default -set xtics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 +set xtics border in scale 1,0.5 mirror rotate by -45 offset character 0, 0, 0 set xtics norangelimit set xtics () set ytics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 @@ -106,7 +107,7 @@ set nox2tics set noy2tics set cbtics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 set cbtics autofreq norangelimit -set title "" +set title "" set title offset character 0, 0, 0 font "" norotate set timestamp bottom set timestamp "" @@ -115,13 +116,13 @@ set rrange [ * : * ] noreverse nowriteback # (currently [8.98847e+307:-8.98847e set trange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) set urange [ * : * ] noreverse nowriteback # (currently [-10.0000:10.0000] ) set vrange [ * : * ] noreverse nowriteback # (currently [-10.0000:10.0000] ) -set xlabel "clients" +set xlabel "Server Type" set xlabel offset character 0, 0, 0 font "" textcolor lt -1 norotate set x2label "" set x2label offset character 0, 0, 0 font "" textcolor lt -1 norotate -set xrange [ * : * ] noreverse nowriteback # (currently [-1.00000:5.00000] ) +set xrange [ 0.5 : 7 ] noreverse nowriteback # (currently [-1.00000:5.00000] ) set x2range [ * : * ] noreverse nowriteback # (currently [-1.00000:5.00000] ) -set ylabel "time(s)" +set ylabel "Time (seconds)" set ylabel offset character 0, 0, 0 font "" textcolor lt -1 rotate by -270 set y2label "" set y2label offset character 0, 0, 0 font "" textcolor lt -1 rotate by -270 @@ -150,5 +151,5 @@ set loadpath set fontpath set fit noerrorvariables GNUTERM = "x11" -plot 'read-hist.data' using 2:xtic(1) title "aesop", '' u 3 ti "thread-per-client", '' u 4 ti "thread-per-client-nb", '' u 5 ti "thread-per-op", '' u 6 ti "thread-pool", '' u 7 ti "event" +plot 'read-time.data' using 4:2:2:3:3:xticlabels(1) notitle with candlesticks # EOF diff --git a/papers/2011/aesop/fig/read-hist.gnuplot b/papers/2011/aesop/fig/write-lat.gnuplot similarity index 91% copy from papers/2011/aesop/fig/read-hist.gnuplot copy to papers/2011/aesop/fig/write-lat.gnuplot index e3ca292..bb12755 100644 --- a/papers/2011/aesop/fig/read-hist.gnuplot +++ b/papers/2011/aesop/fig/write-lat.gnuplot @@ -51,12 +51,13 @@ set key title "" set key inside left top vertical Right noreverse enhanced autotitles box linetype -1 linewidth 1.000 set key noinvert samplen 4 spacing 1 width 0 height 0 set key maxcolumns 0 maxrows 0 +unset key unset label unset arrow set style increment default unset style line unset style arrow -set style histogram clustered gap 2 title offset character 0, 0, 0 +#set style histogram clustered gap 2 title offset character 0, 0, 0 unset logscale set logscale y 10 set offsets 0, 0, 0, 0 @@ -95,7 +96,7 @@ set mztics default set mx2tics default set my2tics default set mcbtics default -set xtics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 +set xtics border in scale 1,0.5 mirror rotate by -45 offset character 0, 0, 0 set xtics norangelimit set xtics () set ytics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 @@ -106,7 +107,7 @@ set nox2tics set noy2tics set cbtics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 set cbtics autofreq norangelimit -set title "" +set title "" set title offset character 0, 0, 0 font "" norotate set timestamp bottom set timestamp "" @@ -115,13 +116,13 @@ set rrange [ * : * ] noreverse nowriteback # (currently [8.98847e+307:-8.98847e set trange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) set urange [ * : * ] noreverse nowriteback # (currently [-10.0000:10.0000] ) set vrange [ * : * ] noreverse nowriteback # (currently [-10.0000:10.0000] ) -set xlabel "clients" +set xlabel "Server Type" set xlabel offset character 0, 0, 0 font "" textcolor lt -1 norotate set x2label "" set x2label offset character 0, 0, 0 font "" textcolor lt -1 norotate -set xrange [ * : * ] noreverse nowriteback # (currently [-1.00000:5.00000] ) +set xrange [ 0.5 : 7 ] noreverse nowriteback # (currently [-1.00000:5.00000] ) set x2range [ * : * ] noreverse nowriteback # (currently [-1.00000:5.00000] ) -set ylabel "time(s)" +set ylabel "Time (seconds)" set ylabel offset character 0, 0, 0 font "" textcolor lt -1 rotate by -270 set y2label "" set y2label offset character 0, 0, 0 font "" textcolor lt -1 rotate by -270 @@ -150,5 +151,5 @@ set loadpath set fontpath set fit noerrorvariables GNUTERM = "x11" -plot 'read-hist.data' using 2:xtic(1) title "aesop", '' u 3 ti "thread-per-client", '' u 4 ti "thread-per-client-nb", '' u 5 ti "thread-per-op", '' u 6 ti "thread-pool", '' u 7 ti "event" +plot 'write-lat.data' using 6:2:3:4:5:xticlabels(1) notitle with candlesticks whiskerbars # EOF diff --git a/papers/2011/aesop/fig/read-hist.gnuplot b/papers/2011/aesop/fig/write-null-lat.gnuplot similarity index 91% copy from papers/2011/aesop/fig/read-hist.gnuplot copy to papers/2011/aesop/fig/write-null-lat.gnuplot index e3ca292..90e5a37 100644 --- a/papers/2011/aesop/fig/read-hist.gnuplot +++ b/papers/2011/aesop/fig/write-null-lat.gnuplot @@ -51,12 +51,13 @@ set key title "" set key inside left top vertical Right noreverse enhanced autotitles box linetype -1 linewidth 1.000 set key noinvert samplen 4 spacing 1 width 0 height 0 set key maxcolumns 0 maxrows 0 +unset key unset label unset arrow set style increment default unset style line unset style arrow -set style histogram clustered gap 2 title offset character 0, 0, 0 +#set style histogram clustered gap 2 title offset character 0, 0, 0 unset logscale set logscale y 10 set offsets 0, 0, 0, 0 @@ -95,7 +96,7 @@ set mztics default set mx2tics default set my2tics default set mcbtics default -set xtics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 +set xtics border in scale 1,0.5 mirror rotate by -45 offset character 0, 0, 0 set xtics norangelimit set xtics () set ytics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 @@ -106,7 +107,7 @@ set nox2tics set noy2tics set cbtics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 set cbtics autofreq norangelimit -set title "" +set title "" set title offset character 0, 0, 0 font "" norotate set timestamp bottom set timestamp "" @@ -115,13 +116,13 @@ set rrange [ * : * ] noreverse nowriteback # (currently [8.98847e+307:-8.98847e set trange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) set urange [ * : * ] noreverse nowriteback # (currently [-10.0000:10.0000] ) set vrange [ * : * ] noreverse nowriteback # (currently [-10.0000:10.0000] ) -set xlabel "clients" +set xlabel "Server Type" set xlabel offset character 0, 0, 0 font "" textcolor lt -1 norotate set x2label "" set x2label offset character 0, 0, 0 font "" textcolor lt -1 norotate -set xrange [ * : * ] noreverse nowriteback # (currently [-1.00000:5.00000] ) +set xrange [ 0.5 : 7 ] noreverse nowriteback # (currently [-1.00000:5.00000] ) set x2range [ * : * ] noreverse nowriteback # (currently [-1.00000:5.00000] ) -set ylabel "time(s)" +set ylabel "Time (seconds)" set ylabel offset character 0, 0, 0 font "" textcolor lt -1 rotate by -270 set y2label "" set y2label offset character 0, 0, 0 font "" textcolor lt -1 rotate by -270 @@ -150,5 +151,5 @@ set loadpath set fontpath set fit noerrorvariables GNUTERM = "x11" -plot 'read-hist.data' using 2:xtic(1) title "aesop", '' u 3 ti "thread-per-client", '' u 4 ti "thread-per-client-nb", '' u 5 ti "thread-per-op", '' u 6 ti "thread-pool", '' u 7 ti "event" +plot 'write-null-lat.data' using 6:2:3:4:5:xticlabels(1) notitle with candlesticks whiskerbars # EOF diff --git a/papers/2011/aesop/fig/read-hist.gnuplot b/papers/2011/aesop/fig/write-null-time.gnuplot similarity index 91% copy from papers/2011/aesop/fig/read-hist.gnuplot copy to papers/2011/aesop/fig/write-null-time.gnuplot index e3ca292..93616bb 100644 --- a/papers/2011/aesop/fig/read-hist.gnuplot +++ b/papers/2011/aesop/fig/write-null-time.gnuplot @@ -51,12 +51,13 @@ set key title "" set key inside left top vertical Right noreverse enhanced autotitles box linetype -1 linewidth 1.000 set key noinvert samplen 4 spacing 1 width 0 height 0 set key maxcolumns 0 maxrows 0 +unset key unset label unset arrow set style increment default unset style line unset style arrow -set style histogram clustered gap 2 title offset character 0, 0, 0 +#set style histogram clustered gap 2 title offset character 0, 0, 0 unset logscale set logscale y 10 set offsets 0, 0, 0, 0 @@ -95,7 +96,7 @@ set mztics default set mx2tics default set my2tics default set mcbtics default -set xtics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 +set xtics border in scale 1,0.5 mirror rotate by -45 offset character 0, 0, 0 set xtics norangelimit set xtics () set ytics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 @@ -106,7 +107,7 @@ set nox2tics set noy2tics set cbtics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 set cbtics autofreq norangelimit -set title "" +set title "" set title offset character 0, 0, 0 font "" norotate set timestamp bottom set timestamp "" @@ -115,13 +116,13 @@ set rrange [ * : * ] noreverse nowriteback # (currently [8.98847e+307:-8.98847e set trange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) set urange [ * : * ] noreverse nowriteback # (currently [-10.0000:10.0000] ) set vrange [ * : * ] noreverse nowriteback # (currently [-10.0000:10.0000] ) -set xlabel "clients" +set xlabel "Server Type" set xlabel offset character 0, 0, 0 font "" textcolor lt -1 norotate set x2label "" set x2label offset character 0, 0, 0 font "" textcolor lt -1 norotate -set xrange [ * : * ] noreverse nowriteback # (currently [-1.00000:5.00000] ) +set xrange [ 0.5 : 7 ] noreverse nowriteback # (currently [-1.00000:5.00000] ) set x2range [ * : * ] noreverse nowriteback # (currently [-1.00000:5.00000] ) -set ylabel "time(s)" +set ylabel "Time (seconds)" set ylabel offset character 0, 0, 0 font "" textcolor lt -1 rotate by -270 set y2label "" set y2label offset character 0, 0, 0 font "" textcolor lt -1 rotate by -270 @@ -150,5 +151,5 @@ set loadpath set fontpath set fit noerrorvariables GNUTERM = "x11" -plot 'read-hist.data' using 2:xtic(1) title "aesop", '' u 3 ti "thread-per-client", '' u 4 ti "thread-per-client-nb", '' u 5 ti "thread-per-op", '' u 6 ti "thread-pool", '' u 7 ti "event" +plot 'write-null-time.data' using 4:2:2:3:3:xticlabels(1) notitle with candlesticks # EOF diff --git a/papers/2011/aesop/fig/read-hist.gnuplot b/papers/2011/aesop/fig/write-time.gnuplot similarity index 91% copy from papers/2011/aesop/fig/read-hist.gnuplot copy to papers/2011/aesop/fig/write-time.gnuplot index e3ca292..0c603a8 100644 --- a/papers/2011/aesop/fig/read-hist.gnuplot +++ b/papers/2011/aesop/fig/write-time.gnuplot @@ -51,12 +51,13 @@ set key title "" set key inside left top vertical Right noreverse enhanced autotitles box linetype -1 linewidth 1.000 set key noinvert samplen 4 spacing 1 width 0 height 0 set key maxcolumns 0 maxrows 0 +unset key unset label unset arrow set style increment default unset style line unset style arrow -set style histogram clustered gap 2 title offset character 0, 0, 0 +#set style histogram clustered gap 2 title offset character 0, 0, 0 unset logscale set logscale y 10 set offsets 0, 0, 0, 0 @@ -95,7 +96,7 @@ set mztics default set mx2tics default set my2tics default set mcbtics default -set xtics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 +set xtics border in scale 1,0.5 mirror rotate by -45 offset character 0, 0, 0 set xtics norangelimit set xtics () set ytics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 @@ -106,7 +107,7 @@ set nox2tics set noy2tics set cbtics border in scale 1,0.5 mirror norotate offset character 0, 0, 0 set cbtics autofreq norangelimit -set title "" +set title "" set title offset character 0, 0, 0 font "" norotate set timestamp bottom set timestamp "" @@ -115,13 +116,13 @@ set rrange [ * : * ] noreverse nowriteback # (currently [8.98847e+307:-8.98847e set trange [ * : * ] noreverse nowriteback # (currently [-5.00000:5.00000] ) set urange [ * : * ] noreverse nowriteback # (currently [-10.0000:10.0000] ) set vrange [ * : * ] noreverse nowriteback # (currently [-10.0000:10.0000] ) -set xlabel "clients" +set xlabel "Server Type" set xlabel offset character 0, 0, 0 font "" textcolor lt -1 norotate set x2label "" set x2label offset character 0, 0, 0 font "" textcolor lt -1 norotate -set xrange [ * : * ] noreverse nowriteback # (currently [-1.00000:5.00000] ) +set xrange [ 0.5 : 7 ] noreverse nowriteback # (currently [-1.00000:5.00000] ) set x2range [ * : * ] noreverse nowriteback # (currently [-1.00000:5.00000] ) -set ylabel "time(s)" +set ylabel "Time (seconds)" set ylabel offset character 0, 0, 0 font "" textcolor lt -1 rotate by -270 set y2label "" set y2label offset character 0, 0, 0 font "" textcolor lt -1 rotate by -270 @@ -150,5 +151,5 @@ set loadpath set fontpath set fit noerrorvariables GNUTERM = "x11" -plot 'read-hist.data' using 2:xtic(1) title "aesop", '' u 3 ti "thread-per-client", '' u 4 ti "thread-per-client-nb", '' u 5 ti "thread-per-op", '' u 6 ti "thread-pool", '' u 7 ti "event" +plot 'write-time.data' using 4:2:2:3:3:xticlabels(1) notitle with candlesticks # EOF hooks/post-receive -- Triton-private Repository

1 0

Triton-private Repository branch, master, updated. 0ac2cdd54bf8715b3a4df4a24acf411b20d52212
by noreply＠mcs.anl.gov 24 Feb '12

24 Feb '12

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "Triton-private Repository". The branch, master has been updated via 0ac2cdd54bf8715b3a4df4a24acf411b20d52212 (commit) from 378c6c7d48db1f7746ee488daf768ca2ab1d0243 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 0ac2cdd54bf8715b3a4df4a24acf411b20d52212 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Fri Feb 24 16:50:28 2012 -0500 change #if block to comment for sloccount fairness ----------------------------------------------------------------------- Summary of changes: .../aesop/src/echo-thread-pool/echo-server-tp.c | 5 +---- 1 files changed, 1 insertions(+), 4 deletions(-) Diff of changes: diff --git a/papers/2011/aesop/src/echo-thread-pool/echo-server-tp.c b/papers/2011/aesop/src/echo-thread-pool/echo-server-tp.c index d42f051..7782ba4 100644 --- a/papers/2011/aesop/src/echo-thread-pool/echo-server-tp.c +++ b/papers/2011/aesop/src/echo-thread-pool/echo-server-tp.c @@ -314,11 +314,8 @@ void threadpool_dispatch (int socket_client, socket_t* socketinfo) /* inserting a the head makes the server work on "hot" requests first * rather than trying to service them fairly in FIFO order. */ -#if 1 SIMPLEQ_INSERT_HEAD(&queue_head, socketinfo, list); -#else - SIMPLEQ_INSERT_TAIL(&queue_head, socketinfo, list); -#endif + /* SIMPLEQ_INSERT_TAIL(&queue_head, socketinfo, list); */ qlen++; assert(num_threads_active > -1); if(num_threads_active == 0) hooks/post-receive -- Triton-private Repository

1 0