July 2014 - Triton-commits - mailman3.cels.anl.gov

branch, master, updated. f755976d4bcb449d335561848c60f927802eff74
by noreply＠mcs.anl.gov 24 Jul '14

24 Jul '14

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via f755976d4bcb449d335561848c60f927802eff74 (commit) from 6d61cf0a727584b367970b41f1bf882b008e4cf7 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit f755976d4bcb449d335561848c60f927802eff74 Author: Shane Snyder <ssnyder(a)mcs.anl.gov> Date: Thu Jul 24 17:06:01 2014 -0500 Final edits before approval, incl. analytic eqs ----------------------------------------------------------------------- Summary of changes: papers/swim/REFERENCES.bib | 9 +++++++++ papers/swim/swim-paper.tex | 36 ++++++++++++++++++++++++++---------- 2 files changed, 35 insertions(+), 10 deletions(-) Diff of changes: diff --git a/papers/swim/REFERENCES.bib b/papers/swim/REFERENCES.bib index c2d9cf2..d605d6c 100644 --- a/papers/swim/REFERENCES.bib +++ b/papers/swim/REFERENCES.bib @@ -237,3 +237,12 @@ publisher={Springer} url = {http://clusterlabs.org/}, note = {Retrieved July 2014} } + +@article{bailey1975, + title={The mathematical theory of infectious diseases and its applications}, + author={BAILEY, NTJ and others}, + journal={The mathematical theory of infectious diseases and its applications. 2nd edition.}, + number={2nd ediition}, + year={1975}, + publisher={Charles Griffin \& Company Ltd 5a Crendon Street, High Wycombe, Bucks HP13 6LE.} +} diff --git a/papers/swim/swim-paper.tex b/papers/swim/swim-paper.tex index 1ac724f..077b18d 100644 --- a/papers/swim/swim-paper.tex +++ b/papers/swim/swim-paper.tex @@ -353,11 +353,12 @@ $t_\mathit{dissem}$ (ms) & $t_\mathit{total}$ (ms) \\ \label{table:converge} \begin{tablenotes} \item[\textdagger] Note: The analytical results here are calculated assuming a 1\% chance -of node failure and packet loss and a subgroup size ($k$) of 1. +of node being faulty, a 1\% chance of packet loss, and a subgroup size ($k$) of 1. \end{tablenotes} \end{threeparttable} \normalsize \end{table*} +%TODO: do we need to have an equation showing how we got analytical average network load? Before evaluating SWIM's performance in the context of a large-scale HPC storage system, we must first select protocol parameters that are @@ -365,9 +366,25 @@ appropriate for that environment. We focus on two key input parameters: the protocol period length ($T'$) and the suspicion timeout in periods ($s$), as we vary the number of members ($n$). These parameters can be used in conjunction with analytical models to calculate the expected time before a fault -is detected by a single member ($t_\mathit{detect}$) as well as +is detected by a single member ($t_{\mathit{detect}}$) as well as the expected time for a given status update to be disseminated to all $n$ members -($t_\mathit{dissem}$). We also further define the total time elapsed from +($t_{\mathit{dissem}}$). $t_{\mathit{detect}}$, originally given in~\cite{Gupta:2001}, +is defined as follows, where $q_f$ is the probability that a node is not faulty: + +\begin{equation*} +t_{\mathit{detect}} = T' \times {1 \over 1 - e^{-q_f}} +\end{equation*} + +$t_{\mathit{dissem}}$ (derived from~\cite{Das:2002}\footnote{Using epidemic theory +from~\cite{bailey1975} and a SWIM specific contact rate given by Das et al.~\cite{Das:2002} in +Section 4.1)} is given as follows, where $n$ is the process group size: + +\begin{equation*} +t_{\mathit{dissem}} = T' \times {n\ln(1/(n - 1)^2) \over 1 - 2n} +\end{equation*} +%TODO: is this equation right? + +We also further define the total time elapsed from the occurrence of a fault to all members being aware of the confirmed failure as: @@ -376,7 +393,7 @@ t_{\mathit{total}} = t_{\mathit{detect}} + (T' \times s) + t_{\mathit{dissem}} \end{equation*} We chose to observe the following constraints in order to select SWIM -parameters (particularly $s$ and $T'$ that are appropriate for HPC storage +parameters (particularly $s$ and $T'$) that are appropriate for HPC storage systems: \begin{itemize} @@ -450,10 +467,8 @@ as of Linux 3.15. % TODO: good reference for this, preferably on IB? % below: Table~\ref{table:converge} shows the impact of varying the protocol period -lengths ($T'$) for a selection of storage system sizes ($n$) ranging from -1,024 to 4,096 servers, given a constant suspicion time ($T' \times s$). The expected -network load and response time values were calculated using analytical equations -from~\cite{Das:2002}. Smaller values +length ($T'$) for a selection of storage system sizes ($n$) ranging from +1,024 to 4,096 servers, given a constant suspicion time ($T' \times s$). Smaller values of $T'$ lead to faster failure detection and dissemination times at the cost of a higher network load. For a given value of $T'$, tolerance to transient failures can be tuned by setting $s$ such that $T' \times s$ is @@ -469,7 +484,7 @@ and 1.53 s, respectively. Further, the average network load imposed by this configuration is still negligible compared to the bandwidth of typical network interconnects in high-performance data centers. A suspicion timeout of 75 protocol period lengths allows the protocol to be resilient to transient -failures of up to 15 seconds, depending on how fast \emph{alive} updates +failures of up to 15 seconds, depending on how fast subsequent \emph{alive} updates are disseminated to the group. This greatly reduces the probability of unnecessary recovery actions, such as rebuilding storage system data. Note that Das et. al.~\cite{Das:2002} recommend a shorter $s$ value of $(3 \lceil \log(n+1) @@ -490,7 +505,7 @@ component may become overwhelmed by the volume of membership updates. However, it is also important to bound the size of this piggyback buffer as part of minimizing the network load imposed by the protocol (in a given time span). For our simulation model, we decide to use a piggyback buffer size of 12, -which yields a total messagesize of 256 bytes if we assume a 64 byte base +which yields a total message size of 256 bytes if we assume a 64 byte base message and 16 bytes per membership update. This message size in conjunction with the network load in Table~\ref{table:converge} produces a network consumption of roughly 2.5 KiB/s per node at 2,048 nodes. @@ -552,6 +567,7 @@ KiB as well as an unexpected decline in performance between 4 MiB and 8 MiB. We note that for a message size of 256 bytes (as used in our example SWIM configuration) the model conservatively over-estimates latency as roughly 6 us rather than the 4 us measured on Tukey. +%TODO: is this right? looks like the model underestimates \subsection{Sensitivity to message loss} \label{sec:results-msg-loss} hooks/post-receive --

1 0

branch, master, updated. 6d61cf0a727584b367970b41f1bf882b008e4cf7
by noreply＠mcs.anl.gov 24 Jul '14

24 Jul '14

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 6d61cf0a727584b367970b41f1bf882b008e4cf7 (commit) from ce90c93ed53bdcedf89bf5795d4d919cf04e4b44 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 6d61cf0a727584b367970b41f1bf882b008e4cf7 Author: Shane Snyder <ssnyder(a)mcs.anl.gov> Date: Thu Jul 24 15:43:26 2014 -0500 remainder of RobR's edits ----------------------------------------------------------------------- Summary of changes: papers/swim/swim-paper.tex | 35 +++++++++++++++++++++-------------- 1 files changed, 21 insertions(+), 14 deletions(-) Diff of changes: diff --git a/papers/swim/swim-paper.tex b/papers/swim/swim-paper.tex index 536f6b8..1ac724f 100644 --- a/papers/swim/swim-paper.tex +++ b/papers/swim/swim-paper.tex @@ -276,7 +276,7 @@ The SWIM protocol semantics differ from more conventional group membership protocols in two notable ways with respect to storage system design. The first is that SWIM does not provide a strongly consistent view of % TODO: robr: be consistent on terminology for participants. Some places we -% are probably calling them members or process group, etc. +% are probably calling them members or process group, nodes, etc. membership among all participants. At any given time, two members may have different views of the system. Secondly, it does not guarantee that updates are disseminated in a consistent order. For example, two @@ -469,7 +469,8 @@ and 1.53 s, respectively. Further, the average network load imposed by this configuration is still negligible compared to the bandwidth of typical network interconnects in high-performance data centers. A suspicion timeout of 75 protocol period lengths allows the protocol to be resilient to transient -failures of up to 15 seconds, greatly reducing the probability of unnecessary +failures of up to 15 seconds, depending on how fast \emph{alive} updates +are disseminated to the group. This greatly reduces the probability of unnecessary recovery actions, such as rebuilding storage system data. Note that Das et. al.~\cite{Das:2002} recommend a shorter $s$ value of $(3 \lceil \log(n+1) \rceil)$, but we extend it in this context to account for shorter period @@ -486,11 +487,11 @@ disseminate (potentially numerous) membership updates throughout the system. This is particularly important in groups where membership is continually changing or in systems with high message loss rates, since the dissemination component may become overwhelmed by the volume of membership updates. -However, it is also important to bound the size of this piggyback buffer to -minimize the network load imposed by the protocol. For our simulation model, -we decide to use a piggyback buffer size of 12, which yields a total message -size of 256 bytes if we assume a 64 byte base message and 16 bytes per -membership update. This message size in conjunction with the +However, it is also important to bound the size of this piggyback buffer as +part of minimizing the network load imposed by the protocol (in a given time span). +For our simulation model, we decide to use a piggyback buffer size of 12, +which yields a total messagesize of 256 bytes if we assume a 64 byte base +message and 16 bytes per membership update. This message size in conjunction with the network load in Table~\ref{table:converge} produces a network consumption of roughly 2.5 KiB/s per node at 2,048 nodes. @@ -597,10 +598,10 @@ a subgroup size of 4, the number of supect nodes declines and the protocol no longer produces any false positives. Finally, with subgroup sizes of 5 or 6 the number of suspected nodes plummets due to decreased likelyhood of all direct and indirect pings -failing for a given target node. We observe that, as expected, the -increased accuracy of the protocol as we increase $k$ results in an -increased network load measured in the average number of messages per node -per second. +failing for a given target node. We observe that, as expected, the +network load imposed by the protocol (measured in the average number +of messages per node per second) scales linearly with $k$, while +the accuracy of the protocol increases exponentially with $k$. \subsection{Validation} \label{sec:results-validate} @@ -651,8 +652,10 @@ the analytical model in practice. One factor contributing to this discrepancy is that we deliberately de-synchronized the period start times for each node by a random amount, meaning that it typically does not take a full $T'=200$ms for a given update to be relayed between two members. We -are also using a round-robin probing strategy that causes wider dispersal on -average than pure random selection of ping targets in each interval. +are also using a more efficient round-robin probing strategy (as suggest by Das et +al.~\cite{Das:2002}) which is not accounted for in the analytical dissemination +time calculation. This round-robin probing causes wider dispersal on average +than pure random selection of ping targets in each interval. These results confirm that both $t_{\mathit{detect}}$ and $t_{\mathit{dissem}}$ are relatively minor components of performance, and @@ -661,7 +664,11 @@ overall performance will be the suspicion timeout $s$. This SWIM configuration, with 4096 servers, should reliably propagate fault notifications to all servers in roughly 17 seconds while still remaining resilient to transient -faults of up to 15 second durations and imposing negligible network load. +faults of up to 15 second durations and imposing negligible network load +(still about 2.5 KiB/s, since the load does not scale with the group size). +Clearly, the constraints from Section~\ref{sec:swim-params} may be modified +to accomodate other group membership use cases, resulting in quicker detection +and dissemination time or reduced sensitivity to transient failures, for instance. \section{Conclusion} \label{sec:conclusion} hooks/post-receive --

1 0

branch, master, updated. ce90c93ed53bdcedf89bf5795d4d919cf04e4b44
by noreply＠mcs.anl.gov 24 Jul '14

24 Jul '14

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via ce90c93ed53bdcedf89bf5795d4d919cf04e4b44 (commit) via 86d47b822f872d4e70bcc2f33fc1e23b1e4d1a8d (commit) from be1f43177eacc71eeafe1e6b8567a6e2916c2ab2 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit ce90c93ed53bdcedf89bf5795d4d919cf04e4b44 Author: Shane Snyder <ssnyder(a)mcs.anl.gov> Date: Thu Jul 24 13:09:17 2014 -0500 Update y axis for message loss graphs commit 86d47b822f872d4e70bcc2f33fc1e23b1e4d1a8d Author: Shane Snyder <ssnyder(a)mcs.anl.gov> Date: Thu Jul 24 13:08:53 2014 -0500 more edits from RobR in analysis and conclusions ----------------------------------------------------------------------- Summary of changes: papers/swim/figs/msg-loss.pdf | Bin 9947 -> 10050 bytes papers/swim/sim_results/msg-loss/msg-loss.plt | 4 +- papers/swim/swim-paper.tex | 36 ++++++++++++++----------- 3 files changed, 22 insertions(+), 18 deletions(-) Diff of changes: diff --git a/papers/swim/figs/msg-loss.pdf b/papers/swim/figs/msg-loss.pdf index aefb88f..e6084f3 100644 Binary files a/papers/swim/figs/msg-loss.pdf and b/papers/swim/figs/msg-loss.pdf differ diff --git a/papers/swim/sim_results/msg-loss/msg-loss.plt b/papers/swim/sim_results/msg-loss/msg-loss.plt index 2c55ef3..5927cc9 100644 --- a/papers/swim/sim_results/msg-loss/msg-loss.plt +++ b/papers/swim/sim_results/msg-loss/msg-loss.plt @@ -47,7 +47,7 @@ set rmargin at screen 0.97 set tmargin at screen 0.60 set bmargin at screen 0.43 set yrange [0:] -set ylabel "max suspect (s)" +set ylabel "max suspect time (s)" plot 'data.dat' using 1:2 notitle w linespoints lt 1 pt 7 linecolor rgb "blue" @@ -71,7 +71,7 @@ set rmargin at screen 0.97 set tmargin at screen 0.23 set bmargin at screen 0.06 set yrange [0:] -set ylabel "pb/msg" +set ylabel "piggyback payload (updates/msg)" set xlabel "k (subgroup size)" offset 0,0.3 plot 'data.dat' using 1:($6/$5) notitle w linespoints lt 1 pt 7 linecolor rgb "blue" diff --git a/papers/swim/swim-paper.tex b/papers/swim/swim-paper.tex index 7aaa6af..536f6b8 100644 --- a/papers/swim/swim-paper.tex +++ b/papers/swim/swim-paper.tex @@ -492,7 +492,7 @@ we decide to use a piggyback buffer size of 12, which yields a total message size of 256 bytes if we assume a 64 byte base message and 16 bytes per membership update. This message size in conjunction with the network load in Table~\ref{table:converge} produces a network consumption of -roughly 2.5 KiB/s per node at 2048 nodes. +roughly 2.5 KiB/s per node at 2,048 nodes. \section{Simulation Analysis} \label{sec:sim-results} @@ -519,7 +519,7 @@ delays. We assume that each node has a full-duplex network card with independent send and receive queues and infinite buffering in the switch complex. The parameters for our LogGP model were obtained by using the \texttt{netgauge} utility~\cite{hoefler2007netgauge} on the -Tukey analysis cluster at the Argonne Leadership Computing Facility. Each +Tukey Linux cluster at the Argonne Leadership Computing Facility. Each Tukey node uses a single-port Mellanox ConnectX 2 QDR Infiniband NIC, and the nodes are interconnected by four Mellanox IS-5600 switches. The \texttt{netgauge} utility assumes that the overhead parameter ($o$) @@ -527,9 +527,9 @@ The \texttt{netgauge} utility assumes that the overhead parameter ($o$) network fabric transmission costs, so we do not apply the $o$ parameter to the communication time calculation. We also take advantage of the fact that \texttt{netgauge} calculates LogGP parameters -independently for a range of message sizes. We use these parameters in a -lookup table to allow the model to more accurately -reflect protocol crossover points and other fabric characteristics. +independently for a range of message sizes, creating a +lookup table to more accurately reflect protocol crossover points and other +fabric characteristics. \begin{figure}[t] \centering @@ -540,6 +540,7 @@ performance on QDR InfiniBand network with MPI.} \label{fig:modelnet} \vspace{-.1in} \end{figure} +%TODO as RobR points out, latency would be better to use here if possible Figure~\ref{fig:modelnet} compares the empirically measured point-to-point bandwidth on the Linux cluster (measured by using \texttt{mpptest}~\cite{gropp1999reproducible}) with a @@ -558,12 +559,15 @@ roughly 6 us rather than the 4 us measured on Tukey. \centering \includegraphics[width=0.48\textwidth]{figs/msg-loss.pdf} \caption{Accuracy and network load metrics for the SWIM protocol executing -for 30 minutes with 2048 members and a message loss probability of 5\%} +for 30 minutes with 2,048 members, a piggyback buffer size of 12, and a message +loss probability of 5\%, all while varying the subgroup size $k$.} \label{fig:msg-loss} \end{figure} +%TODO: this plot needs some magic -- piggyback axis too long, +% also not enough gap between piggyback and message rate graphs To analyze the performance of SWIM in the presence of high message loss rates, -we executed a 30 minute, 2048 member simulation of the SWIM protocol where 5\% +we executed a 30 minute, 2,048 member simulation of the SWIM protocol where 5\% of the total messages are dropped but no member fails completely. This hypothetical scenario would be an extraordinary occurrence in a datacenter environment, but we include it as a demonstration of SWIM behavior in @@ -572,8 +576,8 @@ Figure~\ref{fig:msg-loss} illustrates several performance and accuracy metrics as the SWIM subgroup size $k$ is varied from 1 to 6. The first three figures are accuracy metrics: the number of false positives (i.e. the number of members -falsely confirmed as failed), the number of members suspected as failed, and the -maximum duration any node was suspected. The last two are performance +falsely confirmed as failed), the number of members falsely suspected as failed, +and the maximum duration any node was suspected. The last two are performance metrics: the message rate for each member and the average number of membership updates piggybacked on each message. We gathered the performance metrics from the beginning of the simulation (the @@ -584,7 +588,7 @@ nodes are falsely confirmed as failed. In this configuration, the probability of failed direct and indirect pings generates a large volume of node suspicions that overwhelms the capacity of the dissemination component to correct them. With a -subgroup size of 3, only a single member falsely confirmed as failed, +subgroup size of 3, only a single member is falsely confirmed as failed, though we still observe that nearly all members are suspected at some point. The piggyback buffer size is near capacity (as evidenced by the pb/msg metric) but the increased subgroup size allows the @@ -605,7 +609,8 @@ per second. \centering \includegraphics[width=0.48\textwidth]{figs/validate-graph.pdf} \caption{$t_{\mathit{detect}}$ and $t_{\mathit{dissem}}$ as measured via -simulation using highlighted parameters from Table~\ref{table:converge}. Each box plot shows the minimum, median, maximum, and Q1 and Q3 +simulation using highlighted parameters from Table~\ref{table:converge}. +Each box plot shows the minimum, median, maximum, and Q1 and Q3 quartiles for 15 simulation samples. The predicted values according to analytical models are also shown for comparison.} \label{fig:sim-results} @@ -666,15 +671,14 @@ style epidemic fault detection and group membership protocols for use in large-scale HPC storage systems. We identified a set of characteristics necessary for using eventually consistent group membership protocols like SWIM in HPC storage -systems. - We then used a combination analytical models -and simulation to to select appropriate SWIM parameters for an HPC +systems. We then used a combination of analytical models +and simulation to select appropriate SWIM parameters for an HPC environment while still being tolerant of extraordinary message loss rates. Finally, we performed a study of the SWIM protocol response time as -we varied the number of servers from 128 to 4096 and confirmed that the +we varied the number of servers from 128 to 4,096 and confirmed that the protocol scales well for basic failure cases. We found that the SWIM protocol could be configured to detect and -fully disseminate failure notifications in an exemplar 4096 server storage +fully disseminate failure notifications in an exemplar 4,096 server storage system in roughly 17 seconds, while remaining resilient to transient failures of up to 15 seconds, and imposing a negligible network load. These results suggest that the SWIM protocol is a promising solution for fault detection hooks/post-receive --

1 0

branch, master, updated. be1f43177eacc71eeafe1e6b8567a6e2916c2ab2
by noreply＠mcs.anl.gov 24 Jul '14

24 Jul '14

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via be1f43177eacc71eeafe1e6b8567a6e2916c2ab2 (commit) from 9f9ccefc0252be3a0490f50578d8f822969bbe4a (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit be1f43177eacc71eeafe1e6b8567a6e2916c2ab2 Author: Shane Snyder <ssnyder(a)mcs.anl.gov> Date: Thu Jul 24 12:12:55 2014 -0500 abstract/intro edits from RobR ----------------------------------------------------------------------- Summary of changes: papers/swim/swim-paper.tex | 20 +++++++++++--------- 1 files changed, 11 insertions(+), 9 deletions(-) Diff of changes: diff --git a/papers/swim/swim-paper.tex b/papers/swim/swim-paper.tex index d569f60..7aaa6af 100644 --- a/papers/swim/swim-paper.tex +++ b/papers/swim/swim-paper.tex @@ -95,11 +95,12 @@ protocols for use in large-scale HPC storage systems. This class of algorithms has the potential to increase scalability and decrease fault response time for future production systems. We focus our analysis on the Scalable -Weakly-consistent Infection-style Process Group Membership Protocol (SWIM)~\cite{Das:2002}. We begin by exploring how the +Weakly-consistent Infection-style Process Group Membership Protocol (SWIM)~\cite{Das:2002}. +We begin by exploring how the semantics of this protocol differ from typical HPC group membership -protocols and discuss how those HPC storage systems would need to be -adapted. We then identify how to tune the SWIM protocol -for an HPC environment. Finally, we develop a high-resolution parallel +protocols and discuss how HPC storage systems might need to be +adapted to meet these requirements. We then identify how to tune the SWIM protocol +for deployment in an HPC environment. Finally, we develop a high-resolution parallel discrete event simulation of the protocol to confirm existing analytical models and explore its behavior in greater detail for example system configurations. Our preliminary results indicate that @@ -124,13 +125,14 @@ and accurate fault detection include distributed storage % even when they are contiguously numbered. systems~\cite{Weil:2006,Lakshman:2010} and reliable multicast protocols~\cite{Birman:1993,Birman:1999}. Fault -detection is one component of broader group membership -~\cite{Birman:1993,reiter1996,Das:2002} protocols that are used to maintain +detection is one component of broader group membership protocols +~\cite{Birman:1993,reiter1996,Das:2002} that are used to maintain a global view of available participants as they enter or leave the system. An HPC storage system might use this view to determine the set of available servers for data placement, and changes to the group membership can be used to trigger the re-replication of data in order to maintain resilience. -An inefficient failure detector could lead to data loss +An inefficient failure detector (i.e., one which takes too long to disseminate +failure notifications to the group) could lead to data loss if data is not re-replicated before additional failures occur, while an inaccurate failure detector could lead to unnecessary (and costly) rebuilds of the storage system. @@ -186,8 +188,8 @@ While much of the analysis given in \cite{Das:2002} assumes a widely distributed peer-to-peer environment, we instead evaluate how to adapt SWIM to a horizontally-scalable data center storage environment as would be used for HPC or big data applications. -This environment provides lower latency network capability but also higher -expectations of consistency and responsiveness. +This environment provides lower latency network capability and churn rate but also +higher expectations of consistency and responsiveness. The rest of this paper is organized as follows. In Section \ref{sec:background}, we summarize the SWIM group membership hooks/post-receive --

1 0

branch, master, updated. 9f9ccefc0252be3a0490f50578d8f822969bbe4a
by noreply＠mcs.anl.gov 24 Jul '14

24 Jul '14

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 9f9ccefc0252be3a0490f50578d8f822969bbe4a (commit) via 067f0a672cbe9ae67578f5c8d0e97843d77dde56 (commit) via cec832296740676842b962b9bc75269985999ac5 (commit) from 34236c2f2fda8776adcb3bd097f2081b870c8d79 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 9f9ccefc0252be3a0490f50578d8f822969bbe4a Author: Shane Snyder <ssnyder(a)mcs.anl.gov> Date: Thu Jul 24 10:13:55 2014 -0500 small grammar and clarification edits from RobR commit 067f0a672cbe9ae67578f5c8d0e97843d77dde56 Author: Shane Snyder <ssnyder(a)mcs.anl.gov> Date: Thu Jul 24 00:22:02 2014 -0500 section 2 edits from RobR commit cec832296740676842b962b9bc75269985999ac5 Author: Shane Snyder <ssnyder(a)mcs.anl.gov> Date: Thu Jul 24 00:01:19 2014 -0500 Linux transient network error data from Phil ----------------------------------------------------------------------- Summary of changes: papers/swim/swim-paper.tex | 34 +++++++++++++++++----------------- 1 files changed, 17 insertions(+), 17 deletions(-) Diff of changes: diff --git a/papers/swim/swim-paper.tex b/papers/swim/swim-paper.tex index 80cc690..d569f60 100644 --- a/papers/swim/swim-paper.tex +++ b/papers/swim/swim-paper.tex @@ -237,10 +237,10 @@ group. %protocol periods), which we denote as $s$. After a member is declared as \emph{suspected} by the SWIM failure detector, the protocol continues normal operation -- the \emph{suspected} member may still be selected as a probe -target in future iterations of the protocol. However, if a member $M_i$ remains +target in future iterations of the protocol. However, if a member $M_j$ remains \emph{suspected} for more than $s$ iterations -(i.e. the \emph{suspicion timeout}) of the protocol on member $M_j$, -then $M_j$ will mark the $M_i$ as \emph{failed} and disseminate that +(i.e. the \emph{suspicion timeout}) of the protocol on member $M_i$, +then $M_i$ will mark $M_j$ as \emph{failed} and disseminate that information to other members. %eventually removed from the membership %view of each process in the process group using the dissemination component @@ -262,7 +262,7 @@ gossip-style protocols offer higher efficiency and robustness to failures, although at the cost of a higher dissemination latency. In the SWIM protocol, group membership updates are disseminated by piggybacking this data on the ping and ack messages already generated by the failure -detection protocol. It therefore generates no extra packets and imposes +detection protocol. This dissemination therefore generates no extra packets and imposes minimal additional network load. The information then spreads through the process group as members randomly ping (and ack) each other, ultimately resulting in a complete dissemination of the update. @@ -364,7 +364,7 @@ the protocol period length ($T'$) and the suspicion timeout in periods ($s$), as we vary the number of members ($n$). These parameters can be used in conjunction with analytical models to calculate the expected time before a fault is detected by a single member ($t_\mathit{detect}$) as well as -the time for a given status update to be disseminated to all $n$ members +the expected time for a given status update to be disseminated to all $n$ members ($t_\mathit{dissem}$). We also further define the total time elapsed from the occurrence of a fault to all members being aware of the confirmed failure as: @@ -398,10 +398,10 @@ monitoring interval, which does not include time to reach consensus. % TODO: better reference for specific pacemaker configurations? \item \textbf{Transient Failure Sensitivity}: If the suspicion time in seconds ($T' \times s$) is too short, then the protocol may be susceptible -to false positives due to congestion or transient errors. We propose a goal -of $(T' \times s) \ge 5$ seconds in order to tolerate brief transient -errors, such as those that can be recovered by network component or firmware watchdog -timers. % TODO: good reference for this, preferably on IB? +to false positives due to congestion or transient network card errors. +We propose a goal of $(T' \times s) \ge 10$ seconds, which is long enough +to account for the default network driver transmission timeout of 5 seconds +as of Linux 3.15. % TODO: good reference for this, preferably on IB? \end{itemize} % GLOSSARY of terms @@ -449,7 +449,7 @@ timers. % TODO: good reference for this, preferably on IB? Table~\ref{table:converge} shows the impact of varying the protocol period lengths ($T'$) for a selection of storage system sizes ($n$) ranging from -1024 to 4096 servers, given a constant suspicion time ($T' \times s$). The expected +1,024 to 4,096 servers, given a constant suspicion time ($T' \times s$). The expected network load and response time values were calculated using analytical equations from~\cite{Das:2002}. Smaller values of $T'$ lead to faster failure detection and dissemination times at the @@ -459,8 +459,8 @@ larger than the expected transient failure duration. We can therefore use these parameters to balance performance, network load, and transient failure tolerance. -We selected the example system size (2048 servers) and parameters ($T'=200ms, -s=75)$ highlighted in gray for in-depth analysis via parallel discrete event +We selected the example system size (2,048 servers) and parameters ($T'=200$ ms, +$s=75$) highlighted in gray for in-depth analysis via parallel discrete event simulation. The period length of 200 ms allows us to detect faults and disseminate notifications rapidly in an expected 318 ms and 1.53 s, respectively. Further, the average network load imposed by this @@ -470,16 +470,16 @@ protocol period lengths allows the protocol to be resilient to transient failures of up to 15 seconds, greatly reducing the probability of unnecessary recovery actions, such as rebuilding storage system data. Note that Das et. al.~\cite{Das:2002} recommend a shorter $s$ value of $(3 \lceil \log(n+1) -\rceil)$, but we extend it in this context to account shorter period -intervals ($T'$) will still remaining tolerant of transient failures. +\rceil)$, but we extend it in this context to account for shorter period +intervals ($T'$) while still remaining tolerant of transient failures. Finally, the total -time needed for the protocol to reach global consensus on a failed member is +time expected for the protocol to reach global consensus on a failed member is approximately 17 seconds. This combination of parameters readily meets the constraints described at the beginning of this section. One detail of the SWIM protocol that has been neglected thus far is the number of membership updates to piggyback on each ping and ack message. It is -critical that this piggyback buffer size is large enough to effectively +critical that this piggyback buffer be large enough to effectively disseminate (potentially numerous) membership updates throughout the system. This is particularly important in groups where membership is continually changing or in systems with high message loss rates, since the dissemination @@ -489,7 +489,7 @@ minimize the network load imposed by the protocol. For our simulation model, we decide to use a piggyback buffer size of 12, which yields a total message size of 256 bytes if we assume a 64 byte base message and 16 bytes per membership update. This message size in conjunction with the -network load in Table~\ref{table:converge} produces a network consumption or +network load in Table~\ref{table:converge} produces a network consumption of roughly 2.5 KiB/s per node at 2048 nodes. \section{Simulation Analysis} hooks/post-receive --

1 0

branch, master, updated. 34236c2f2fda8776adcb3bd097f2081b870c8d79
by noreply＠mcs.anl.gov 23 Jul '14

23 Jul '14

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 34236c2f2fda8776adcb3bd097f2081b870c8d79 (commit) from c991490570a2f20ad54b05996c145316d9e08e9f (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 34236c2f2fda8776adcb3bd097f2081b870c8d79 Author: Rob Ross <rbross(a)mailinator.com> Date: Wed Jul 23 15:08:19 2014 -0500 some notes on subgroups and lost messages and probability of false suspects. ----------------------------------------------------------------------- Summary of changes: papers/swim/appendix.tex | 32 ++++++++++++++++++++++++++++++++ 1 files changed, 32 insertions(+), 0 deletions(-) create mode 100644 papers/swim/appendix.tex Diff of changes: diff --git a/papers/swim/appendix.tex b/papers/swim/appendix.tex new file mode 100644 index 0000000..236a2c5 --- /dev/null +++ b/papers/swim/appendix.tex @@ -0,0 +1,32 @@ +\section*{Equations} + +Assume a probability that a message is dropped $P_{drop}$ and a subgroup size $k$. + +The probability of a failed direct ping is simply: +\begin{equation} +P_{fail_d} = P_{drop} + (1 - P_{drop}) \times P_{drop} +\end{equation} + +The probability of a failed indirect ping is: +\begin{equation} +\begin{split} +P_{fail_i} = P_{drop} + (1 - P_{drop}) \times P_{drop} + + (1 - P_{drop})^2 \times P_{drop} \\ + + (1 - P_{drop})^3 \times P_{drop} +\end{split} +\end{equation} + +The probability of a false suspicion is then: +\begin{equation} +P_{false} = P_{fail_d} \times P_{fail_i}^k +\end{equation} + +Some example values: +\begin{tabular}{ll} +k & $P_{false}$ \\ +2 & .0033 \\ +3 & .00061 \\ +4 & .00011 \\ +5 & .000021 \\ +\end{tabular} + hooks/post-receive --

1 0

branch, master, updated. c991490570a2f20ad54b05996c145316d9e08e9f
by noreply＠mcs.anl.gov 23 Jul '14

23 Jul '14

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via c991490570a2f20ad54b05996c145316d9e08e9f (commit) from 7e5e91414faa8614fcd083d4f3b62d07e63aa158 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit c991490570a2f20ad54b05996c145316d9e08e9f Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jul 23 13:27:12 2014 -0400 edit msg loss analysis ----------------------------------------------------------------------- Summary of changes: papers/swim/swim-paper.tex | 67 +++++++++++++++++++++++--------------------- 1 files changed, 35 insertions(+), 32 deletions(-) Diff of changes: diff --git a/papers/swim/swim-paper.tex b/papers/swim/swim-paper.tex index 36feb3e..80cc690 100644 --- a/papers/swim/swim-paper.tex +++ b/papers/swim/swim-paper.tex @@ -539,8 +539,7 @@ performance on QDR InfiniBand network with MPI.} \vspace{-.1in} \end{figure} -Figure~\ref{fig:modelnet} compares the empirically measured point-to-point -bandwidth on the Linux cluster (measured by using +Figure~\ref{fig:modelnet} compares the empirically measured point-to-point bandwidth on the Linux cluster (measured by using \texttt{mpptest}~\cite{gropp1999reproducible}) with a simulation of the point-to-point performance using our simulation framework. We see that the simulated performance closely matches the performance trends @@ -562,36 +561,40 @@ for 30 minutes with 2048 members and a message loss probability of 5\%} \end{figure} To analyze the performance of SWIM in the presence of high message loss rates, -we set up a 30 minute, 2048 member simulation of the SWIM protocol where 5\% -of the total messages are dropped but no member fails completely. Note that -this scenario would be an extraordinary occurrence in a datacenter -environment (likely causing loss of availability in numerous distributed -components), but we include it as a demonstration of how the SWIM protocol -would operate in extreme environments. -Figure~\ref{fig:msg-loss} illustrates the behavior of numerous failure detection -performance metrics as the SWIM subgroup size $k$ is varied from 1 to 6 members. -These metrics include: the number of false positives (i.e. the number of members -falsely confirmed as failed), the number of members suspected as failed, the -maximum duration a node was suspected, the message rate for each member, and -the average number of membership updates contained in each message. - -It can be seen that at lower values of $k$ (e.g., 1 or 2) protocol performance -is particularly poor -- almost all nodes are falsely confirmed as failed. -This is due to the large quantity of suspected nodes caused by the high message -loss rate and the low value of $k$. This large number of suspected nodes quickly -overwhelms the capacity of the piggyback buffer, preventing the dissemination -component from effectively propagating updates throughout the system. With a -subgroup size of 3, there is only a single member falsely confirmed as failed -(though nearly all members are suspected at some point). Even though the average -piggyback buffer size is near capacity, the increased subgroup size allows the -dissemination component to more effectively propagate membership updates after a -message loss. At a subgroup size of 4, the protocol crosses over and no longer -exhibits any false positives, due mostly to the decrease in suspected nodes. -Finally, with subgroup sizes of 5 or 6 the number of suspected nodes plummets, -as the probability of a direct ping and all indirect pings being lost becomes -nearly negligible. Note that, as expected, the increased accuracy of the protocol -in this failure scenario comes at the cost of an increased network load due -to the use of larger subgroup sizes. +we executed a 30 minute, 2048 member simulation of the SWIM protocol where 5\% +of the total messages are dropped but no member fails completely. +This hypothetical scenario would be an extraordinary occurrence in a datacenter +environment, but we include it as a demonstration of SWIM behavior in +extreme conditions. +Figure~\ref{fig:msg-loss} illustrates several performance and accuracy +metrics as the SWIM subgroup size $k$ is varied from 1 to 6. +The first three figures are accuracy metrics: +the number of false positives (i.e. the number of members +falsely confirmed as failed), the number of members suspected as failed, and the +maximum duration any node was suspected. The last two are performance +metrics: the message rate for each member and +the average number of membership updates piggybacked on each message. We +gathered the performance metrics from the beginning of the simulation (the +first 15 to 30 seconds) before any false positives were generated. + +Protocol accuracy is particularly poor at $k$ values of 1 or 2 -- almost all +nodes are falsely confirmed as failed. In this configuration, the +probability of failed direct and indirect pings generates a large volume of node +suspicions that overwhelms the capacity of the dissemination component to +correct them. With a +subgroup size of 3, only a single member falsely confirmed as failed, +though we still observe that nearly all members are suspected at some point. +The piggyback buffer size is near capacity (as evidenced by the pb/msg +metric) but the increased subgroup size allows the +dissemination component to more effectively propagate membership updates. At +a subgroup size of 4, the number of supect nodes declines and the +protocol no longer produces any false positives. +Finally, with subgroup sizes of 5 or 6 the number of suspected nodes +plummets due to decreased likelyhood of all direct and indirect pings +failing for a given target node. We observe that, as expected, the +increased accuracy of the protocol as we increase $k$ results in an +increased network load measured in the average number of messages per node +per second. \subsection{Validation} \label{sec:results-validate} hooks/post-receive --

1 0

branch, master, updated. 7e5e91414faa8614fcd083d4f3b62d07e63aa158
by noreply＠mcs.anl.gov 23 Jul '14

23 Jul '14

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 7e5e91414faa8614fcd083d4f3b62d07e63aa158 (commit) from a2b1097378dc1bbf3a671844dce63106abf6c9e3 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 7e5e91414faa8614fcd083d4f3b62d07e63aa158 Author: Shane Snyder <ssnyder(a)mcs.anl.gov> Date: Wed Jul 23 11:58:42 2014 -0500 Update table 1 to use constant suspicion time ----------------------------------------------------------------------- Summary of changes: papers/swim/swim-paper.tex | 26 +++++++++++++------------- 1 files changed, 13 insertions(+), 13 deletions(-) Diff of changes: diff --git a/papers/swim/swim-paper.tex b/papers/swim/swim-paper.tex index b358d4e..36feb3e 100644 --- a/papers/swim/swim-paper.tex +++ b/papers/swim/swim-paper.tex @@ -330,20 +330,20 @@ $n$ & $T'$ (ms) & $s$ & per member) & $t_\mathit{detect}$ (ms) & $T' \times s$ (ms) & $t_\mathit{dissem}$ (ms) & $t_\mathit{total}$ (ms) \\ \hline -1024 & \multirow{3}{*}{.10} & \multirow{3}{*}{75} & \multirow{3}{*}{20976.0} & -\multirow{3}{*}{.16} & \multirow{3}{*}{7.50} & .69 & 8.35 \\ -2048 & & & & & & .76 & 8.42 \\ -4096 & & & & & & .83 & 8.49 \\ +1024 & \multirow{3}{*}{.10} & \multirow{3}{*}{150000} & \multirow{3}{*}{20976.0} & +\multirow{3}{*}{.16} & \multirow{3}{*}{15000.00} & .69 & 15000.85 \\ +2048 & & & & & & .76 & 15000.92 \\ +4096 & & & & & & .83 & 15000.99 \\ \hline -1024 & \multirow{3}{*}{10.00} & \multirow{3}{*}{75} & \multirow{3}{*}{209.8} & -\multirow{3}{*}{15.91} & \multirow{3}{*}{750.00} & 69.34 & 835.25 \\ -2048 & & & & & & 76.26 & 842.17 \\ -4096 & & & & & & 83.19 & 849.10 \\ +1024 & \multirow{3}{*}{10.00} & \multirow{3}{*}{1500} & \multirow{3}{*}{209.8} & +\multirow{3}{*}{15.91} & \multirow{3}{*}{15000.00} & 69.34 & 15085.25 \\ +2048 & & & & & & 76.26 & 15092.17 \\ +4096 & & & & & & 83.19 & 15099.10 \\ \hline -1024 & \multirow{3}{*}{1000.00} & \multirow{3}{*}{75} & \multirow{3}{*}{2.1} & -\multirow{3}{*}{1591.28} & \multirow{3}{*}{75000.00} & 6933.88 & 83525.16 \\ -2048 & & & & & & 7625.99 & 84217.27 \\ -4096 & & & & & & 8318.54 & 84909.82 \\ +1024 & \multirow{3}{*}{1000.00} & \multirow{3}{*}{15} & \multirow{3}{*}{2.1} & +\multirow{3}{*}{1591.28} & \multirow{3}{*}{15000.00} & 6933.88 & 23525.16 \\ +2048 & & & & & & 7625.99 & 24217.27 \\ +4096 & & & & & & 8318.54 & 24909.82 \\ \hline \rowcolor{lightgray} 2048 & 200.00 & 75 & 10.5 & 318.26 & 15000.00 & 1525.20 & 16843.46 \\ \hline @@ -449,7 +449,7 @@ timers. % TODO: good reference for this, preferably on IB? Table~\ref{table:converge} shows the impact of varying the protocol period lengths ($T'$) for a selection of storage system sizes ($n$) ranging from -1024 to 4096 servers, given a constant suspect timeout ($s$). The expected +1024 to 4096 servers, given a constant suspicion time ($T' \times s$). The expected network load and response time values were calculated using analytical equations from~\cite{Das:2002}. Smaller values of $T'$ lead to faster failure detection and dissemination times at the hooks/post-receive --

1 0

branch, master, updated. a2b1097378dc1bbf3a671844dce63106abf6c9e3
by noreply＠mcs.anl.gov 23 Jul '14

23 Jul '14

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via a2b1097378dc1bbf3a671844dce63106abf6c9e3 (commit) from 1343af10a54c13f8a22b9a646c4c0c41364d6efe (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit a2b1097378dc1bbf3a671844dce63106abf6c9e3 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jul 23 12:36:52 2014 -0400 spell check ----------------------------------------------------------------------- Summary of changes: papers/swim/swim-paper.tex | 24 ++++++++++++------------ 1 files changed, 12 insertions(+), 12 deletions(-) Diff of changes: diff --git a/papers/swim/swim-paper.tex b/papers/swim/swim-paper.tex index 8d20ab2..b358d4e 100644 --- a/papers/swim/swim-paper.tex +++ b/papers/swim/swim-paper.tex @@ -153,7 +153,7 @@ and efficient detection of failures~\cite{Gupta:2001}. % the evaluation." In practice, failure detector implementations usually divide systems into smaller groups with independent -failure domains (introducing artificial limitiations on the +failure domains (introducing artificial limitations on the range of failures the system can tolerate) or delegate group membership maintenance to a specialized subset of processes (increasing engineering complexity and failing to leverage the full network capacity of the system). @@ -366,7 +366,7 @@ conjunction with analytical models to calculate the expected time before a fault is detected by a single member ($t_\mathit{detect}$) as well as the time for a given status update to be disseminated to all $n$ members ($t_\mathit{dissem}$). We also further define the total time elapsed from -the occurance of a fault to all members being aware of the confirmed failure +the occurrence of a fault to all members being aware of the confirmed failure as: \begin{equation*} @@ -397,7 +397,7 @@ such as Pacemaker~\cite{pacemaker-web} are often deployed with a 30 second monitoring interval, which does not include time to reach consensus. % TODO: better reference for specific pacemaker configurations? \item \textbf{Transient Failure Sensitivity}: If the suspicion time in -seconds ($T' \times s$) is too short, then the protocol may be succeptible +seconds ($T' \times s$) is too short, then the protocol may be susceptible to false positives due to congestion or transient errors. We propose a goal of $(T' \times s) \ge 5$ seconds in order to tolerate brief transient errors, such as those that can be recovered by network component or firmware watchdog @@ -454,7 +454,7 @@ network load and response time values were calculated using analytical equations from~\cite{Das:2002}. Smaller values of $T'$ lead to faster failure detection and dissemination times at the cost of a higher network load. For a given value of $T'$, tolerance to -transient falures can be tuned by setting $s$ such that $T' \times s$ is +transient failures can be tuned by setting $s$ such that $T' \times s$ is larger than the expected transient failure duration. We can therefore use these parameters to balance performance, network load, and transient failure tolerance. @@ -464,7 +464,7 @@ s=75)$ highlighted in gray for in-depth analysis via parallel discrete event simulation. The period length of 200 ms allows us to detect faults and disseminate notifications rapidly in an expected 318 ms and 1.53 s, respectively. Further, the average network load imposed by this -configuration is still negligible compared to the bandwith of typical network +configuration is still negligible compared to the bandwidth of typical network interconnects in high-performance data centers. A suspicion timeout of 75 protocol period lengths allows the protocol to be resilient to transient failures of up to 15 seconds, greatly reducing the probability of unnecessary @@ -564,7 +564,7 @@ for 30 minutes with 2048 members and a message loss probability of 5\%} To analyze the performance of SWIM in the presence of high message loss rates, we set up a 30 minute, 2048 member simulation of the SWIM protocol where 5\% of the total messages are dropped but no member fails completely. Note that -this scenario would be an extraordinary occurence in a datacenter +this scenario would be an extraordinary occurrence in a datacenter environment (likely causing loss of availability in numerous distributed components), but we include it as a demonstration of how the SWIM protocol would operate in extreme environments. @@ -584,7 +584,7 @@ component from effectively propagating updates throughout the system. With a subgroup size of 3, there is only a single member falsely confirmed as failed (though nearly all members are suspected at some point). Even though the average piggyback buffer size is near capacity, the increased subgroup size allows the -dissemination component to more effectivly propagate membership updates after a +dissemination component to more effectively propagate membership updates after a message loss. At a subgroup size of 4, the protocol crosses over and no longer exhibits any false positives, due mostly to the decrease in suspected nodes. Finally, with subgroup sizes of 5 or 6 the number of suspected nodes plummets, @@ -622,7 +622,7 @@ Figure \ref{fig:sim-results} compares the performance measured via simulation to expected values based on the analytical models. To be concise, we only consider the time taken to detect a fault ($t_{\mathit{detect}}$) and the time to disseminate updates to all -members ($t_{\mathit{dissem}}$). The overall time from fault occurance to +members ($t_{\mathit{dissem}}$). The overall time from fault occurrence to global convergence ($t_{\mathit{total}}$) is dominated by the suspicion time $T' \times s$, which is a fixed value. We observe that $t_{\mathit{detect}}$ remains roughly constant regardless of scale, but that in practice the @@ -665,14 +665,14 @@ systems. We then used a combination analytical models and simulation to to select appropriate SWIM parameters for an HPC environment while still being tolerant of extraordinary message loss rates. -Finally, we performed a study of the SWIM protcol response time as +Finally, we performed a study of the SWIM protocol response time as we varied the number of servers from 128 to 4096 and confirmed that the protocol scales well for basic failure cases. We found that the SWIM protocol could be configured to detect and -fully disseminate failure notifications in an examplar 4096 server storage +fully disseminate failure notifications in an exemplar 4096 server storage system in roughly 17 seconds, while remaining resilient to transient failures of up to 15 seconds, and imposing a negligible network load. These results -suggest that the SWIM protocol is a promising solution for fault detectin +suggest that the SWIM protocol is a promising solution for fault detection and group membership in future HPC storage architectures. In future work we intend to analyze more complex, statistically generated @@ -683,7 +683,7 @@ head-to-head comparisons. %%%%%%%%%%%%%%%%%%%%% %% Acknowledgements %% %%%%%%%%%%%%%%%%%%%%% -\section*{Acknowledgements} +\section*{Acknowledgments} This research was supported by the United States Department of Defense. This work was also supported by the Department of Energy, Office of Science, hooks/post-receive --

1 0

branch, master, updated. 1343af10a54c13f8a22b9a646c4c0c41364d6efe
by noreply＠mcs.anl.gov 23 Jul '14

23 Jul '14

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 1343af10a54c13f8a22b9a646c4c0c41364d6efe (commit) from e7b3a4c1af9a240544c44c7b9193ff14500314a3 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 1343af10a54c13f8a22b9a646c4c0c41364d6efe Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Jul 23 12:31:46 2014 -0400 fix colors in graph again ----------------------------------------------------------------------- Summary of changes: papers/swim/figs/validate-graph.pdf | Bin 8156 -> 8147 bytes .../validate-graph-data/validate-graph.plt | 8 ++++---- 2 files changed, 4 insertions(+), 4 deletions(-) Diff of changes: diff --git a/papers/swim/figs/validate-graph.pdf b/papers/swim/figs/validate-graph.pdf index 3063100..fe321f0 100644 Binary files a/papers/swim/figs/validate-graph.pdf and b/papers/swim/figs/validate-graph.pdf differ diff --git a/papers/swim/sim_results/validate-graph-data/validate-graph.plt b/papers/swim/sim_results/validate-graph-data/validate-graph.plt index 95d42fc..53f7397 100644 --- a/papers/swim/sim_results/validate-graph-data/validate-graph.plt +++ b/papers/swim/sim_results/validate-graph-data/validate-graph.plt @@ -32,9 +32,9 @@ set bmargin at screen 0.58 set tmargin at screen 0.98 set ylabel "t_detect (s)" offset 1 -plot 't_detect.box.dat' using 1:3:2:7:6 with candlesticks lw 2 linecolor rgb "blue" notitle whiskerbars, \ +plot 't_detect.box.dat' using 1:3:2:7:6 with candlesticks lw 2 linecolor rgb "black" notitle whiskerbars, \ '' using 1:4:4:4:4 with candlesticks lw 2 linecolor rgb "black" notitle, \ -'analytic.dat' using 1:2 title "expected (analytical model)" w linespoints lt 1 pt 7 linecolor rgb "red" +'analytic.dat' using 1:2 title "expected (analytical model)" w linespoints lt 1 pt 7 linecolor rgb "blue" #'' using 1:4 with lines lw 1 linecolor rgb "grey" title "median" #set xtics format "%g" @@ -52,9 +52,9 @@ set tmargin at screen 0.53 set yrange [0:2] set xlabel "number of servers" offset 0,0.3 -plot 't_dissem.box.dat' using 1:3:2:7:6 with candlesticks lw 2 linecolor rgb "blue" notitle whiskerbars, \ +plot 't_dissem.box.dat' using 1:3:2:7:6 with candlesticks lw 2 linecolor rgb "black" notitle whiskerbars, \ '' using 1:4:4:4:4 with candlesticks lw 2 linecolor rgb "black" notitle, \ -'analytic.dat' using 1:3 notitle w linespoints lt 1 pt 7 linecolor rgb "red" +'analytic.dat' using 1:3 notitle w linespoints lt 1 pt 7 linecolor rgb "blue" # '' using 1:4 with lines lw 1 linecolor rgb "grey" title "median" unset multiplot hooks/post-receive --

1 0