March 2013 - Triton-commits - mailman3.cels.anl.gov

branch, master, updated. 8a1778de45a2d25395c43fff868bb155032dbf8b
by noreply＠mcs.anl.gov 28 Mar '13

28 Mar '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, master has been updated via 8a1778de45a2d25395c43fff868bb155032dbf8b (commit) from 6953027aa8d690df2ffb051cbeef3955598fe6bf (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 8a1778de45a2d25395c43fff868bb155032dbf8b Author: cengiz <cengiz.k(a)uconn.edu> Date: Fri Mar 29 00:16:44 2013 -0400 Updated the abstract * The abstract that we agreed on Mar 22 meeting ----------------------------------------------------------------------- Summary of changes: papers/asg-osd/paper.tex | 1339 +++++++++++++++++++++++++++++----------------- 1 files changed, 857 insertions(+), 482 deletions(-) Diff of changes: diff --git a/papers/asg-osd/paper.tex b/papers/asg-osd/paper.tex index 7a2c434..0b73a8d 100644 --- a/papers/asg-osd/paper.tex +++ b/papers/asg-osd/paper.tex @@ -1,482 +1,857 @@ -% TEMPLATE for Usenix papers, specifically to meet requirements of -% USENIX '05 -% originally a template for producing IEEE-format articles using LaTeX. -% written by Matthew Ward, CS Department, Worcester Polytechnic Institute. -% adapted by David Beazley for his excellent SWIG paper in Proceedings, -% Tcl 96 -% turned into a smartass generic template by De Clarke, with thanks to -% both the above pioneers -% use at your own risk. Complaints to /dev/null. -% make it two column with no page numbering, default is 10 point - -% Munged by Fred Douglis <douglis(a)research.att.com> 10/97 to separate -% the .sty file from the LaTeX source template, so that people can -% more easily include the .sty file into an existing document. Also -% changed to more closely follow the style guidelines as represented -% by the Word sample file. - -% Note that since 2010, USENIX does not require endnotes. If you want -% foot of page notes, don't include the endnotes package in the -% usepackage command, below. - -% This version uses the latex2e styles, not the very ancient 2.09 stuff. -\documentclass[letterpaper,twocolumn,10pt]{article} -\usepackage{usenix,epsfig,endnotes} -\begin{document} - -%don't want date printed -\date{} - -%make title bold and 14 pt font (Latex default is non-bold, 16 pt) -\title{\Large \bf Concept of an Object-Based Storage System based on Conditional Operations and Versioning} - -%for single author (just remove % characters) -%\author{ -%{\rm Your N.\ Here}\\ -%Your Institution -%\and -%{\rm Second Name}\\ -%Second Institution -% copy the following lines to add more authors -% \and -% {\rm Name}\\ -%Name Institution -%} % end author - -\maketitle - -% Use the following at camera-ready time to suppress page numbers. -% Comment it out when you first submit the paper for review. -\thispagestyle{empty} - - -\subsection*{Abstract} -Object-based storage model offers to store and access data using an \textit{object} based -interface instead of using traditional \textit{blocks} and in addition to self-management -capabilities it also provides easier data sharing and better security compared to traditional -storage interfaces. Previous work tried to improve various aspects of the object-based storage -model by building software abstractions of it atop actual block-based devices. -In this paper, we introduce the concept of a new object-based storage model, ASG, that unifies -the \textit{conditional operation} and ~\textit{versioning} features previously explored in related studies. -We explain the fundamental properties of ASG storage model and compare it with another -object-based storage model to emphasize its strenghts and weaknesses. We also present -a namespace implementation built using ASG to validate the concepts of this new -model. - -\section{Introduction} - -Storage devices have seen a lot of improvement in terms of performance and complexity in the last decades; -but they could not catch the development rate of the computation and processing units of the computers. -In order to improve the performance of storage devices, optimizations, such as parallel computing, have -been introduced. However, any benefit from these optimizations are limited by the current interfaces of the -storage interfaces; such as SCSI, IDE or ATA. These interfaces stayed mostly unmodified in recent years -and therefore are not capable of meeting the demands of storage applications anymore such as -high-performance, security and portability. In order to meet these demands, a new storage model called -\textit{object-based storage} has been developed ~\cite{1222722, osd-3}. Storage devices that store and -access the data according to the object-based storage model are called as \textit{object-based storage -devices}; however many file systems prefer to build software implementations of object-based storage model -on block-based devices, rather than using object-based storage devices. - -An \textit{object} is an ordered logical collection of bytes with a numerical identifier. Objects consist -of data, user settable attributes and device managed metadata including security policies that protect data -~\cite{1222722, Welch08scalableperformance}. They have variable sizes and they can be used to store any -kind of data in the storage system. Having data organized as objects makes it possible for the -disk to do data management operations on behalf of the user ~\cite{5944457} and offloads the storage -management from the host operating system to the object-based storage. User-settable attributes can -be further enriched to set performance goals, manage replicas, or handle concurrency in object-based -storage model to enable self-management capabilities and they give the user an effective control of data -on the disk~\cite{gibson:storage, 5944457}. Object-based storage is also suitable for realizing -\textit{active storage}~\cite{5944457,Acharya:1998:ADP:291006.291026, riedel:thesis}; because objects -provide more features to encapsulate the computations compared to traditional blocks. - -Although object-based storage model has been investigated in many research studies and several -optimizations have been introduced for this model, versioning and conditional operations in -object-based storage have been covered in only limited number of efforts. \textit{Ursa Minor} is a parallel -file system supporting write versioning; however, \textit{Ursa Minor} keeps the existing object-storage -interface~\cite{osd-3} mostly intact except for introducing \textit{slices} which are fragments of object -data and it uses timestamps to distinguish different versions of data. Transactional Object Storage Device -(TOSD) work~\cite{carns2010object} shows that object-based storage is a common component of many parallel -file systems and it introduces three optimizations to the object-based storage model in order to serve -highly concurrent workloads better; atomicity, versioning and commutativity. Goodell et al. -~\cite{goodell-pdsw12} extended the POSIX API by organizing the storage around data objects in order to map -complex data structures to these data objects and have direct access between the data objects and -applications. Carns et al. ~\cite{carns-pdsw12} investigated conditional update operations as an -alternative to distributed pessimistic locking operations in object-based storage systems. - -We believe an object-based storage system can offer a lot more sophisticated functionalities by -implementing versioning and conditional operations and we introduce a new object-based storage -system model, ASG, that unifies the conditional operation and versioning features previously explored in -related studies described in the previous paragraph. The rest of the paper is organized as follows; -Section 2 talks about related work in object-based storage systems. Section 3 describes the ASG -architecture in details. Section 4 compares the ASG model with existing T10 model and identifies the -advantages\&disadvantages of each model. We give a sample usage scenario of ASG model in Section 5 followed -by the conclusion and possible directions for future development of ASG in Section 6. - -\section{Related Work} - -Network-Attached Secure Disk (NASD) ~\cite{gibson:storage} from Carnegie Mellon is the primary study on -object-based storage systems and it led to specifications of standards~\cite{osd-3} for object-based -storage devices. It is based on the idea of integrating the disk management functions into the storage -device and it offers a variable-length object-storage interface. NASD introduces variable-length objects -with attributes, rather than fixed-length traditional blocks, to enable self-management and to obviate the -need to know about the host operating system. Moving data management to the storage disks increases the -networking, security and space management capabilities. - -OSD+~\cite{springerlink:10.1007/s10766-012-0207-8,10.1109/SBAC-PAD.2011.12} presents a model very similar to -the one specified by the OSD standard ~\cite{osd-3} except for the addition of dedicated directory objects. -The directory objects in OSD+ store file names and attributes and support metadata related operations. Each -OSD+ stores a single directory object. In order to guarantee correctness and atomicity, OSD+ relies on the -back-end file system (Ext3, Ext4) and three-phase commit protocol~\cite{Skeen:1983:FMC:1313337.1313750}. - -Panasas File System~\cite{Welch08scalableperformance} is built on object-based storage devices. Panasas -File System clients have direct access to the objects which are a combination of data and attributes. OSD -wire protocol of Panasas uses the operations from the OSD standard~\cite{osd-3} to enable byte-oriented -access to data, to manipulate attributes and to create or delete objects. Panasas clients use iSCSI/OSD -protocol to access the object-based storage devices. - -Lustre ~\cite{lustre-web} is a distributed file system based on object-based storage. Lustre has two types -of servers; Metadata Server (MDS) and Object Storage Server (OSS). Actual metadata is stored on Metadata -Targets (MDTs) and file data is stored on Object Storage Targets (OSTs) as objects. MDS is responsible -for handling metadata requests and making the metadata stored in MDTs available to the clients; whereas -OSS is responsible for filesystem I/O and for providing access to file data stored on OSTs. - -Other related work~\cite{Devulapalli07integratingparallel} from Ohio Supercomputing Center looked at -mapping Parallel Virtual File System~\cite{pvfs-web} on top of an existing object-based storage -emulation~\cite{1306916}. This mapping moved the functionality of the common components of a traditional -storage system; such as I/O, directory or metadata servers to OSDs and improved the performance -of the overall system due to the capabilities of the object-based storage devices.~\cite{OSDCluster08, OSDPDSI08, 10.1109/SNAPI.2008.14} - -\section{Implementation Strategies} -\label{primitives} -In this section we describe the ASG storage model, its fundamental building blocks and basic primitives. - -\subsection{Architecture} -The main architecture of ASG storage model is shown in Figure \ref{architecture}. - -\begin{figure}[h] -\centering -\includegraphics[width=\columnwidth, keepaspectratio]{overview.pdf} -\caption{Architecture of ASG Storage Model} -\label{architecture} -\end{figure} - -\begin{itemize} -\item The basic building block of the ASG storage model is a \textit{record}. Each record includes a key, a -version number, data and length of data. The key, version number and length of data are represented with -integers; whereas as data is a variable length array of bytes. Key is the numerical identifier of a record. -Version numbers are used to order writes to the record. Data field can be empty and a record at its -\textit{initial condition} will have version number \textit{zero} and will contain no data. -\item Fork is a collection of records and it provides a distinct namespace for the records it contains. Each -fork is identified by a fixed-length integer. -\item Object is a collection of forks and it provides a distinct namespace for the forks it contains. Each -object is identified by a fixed-length integer. -\item Container is a collection of objects and it provides a distinct namespace for the objects it contains. -Each container is identified by a fixed-length integer. Containers partition the storage system into logical -units; as an example each container could contain a distinct file system. -\end{itemize} - -The record, fork and object identifiers in ASG storage model are not global. For example; two different -containers can have objects with the same identifiers. Similarly, two different objects/forks can have -forks/records with the same identifiers. - -\subsection{Operations} -In this section we describe the ASG storage model operations a client use to interact with the storage -system. - -\subsubsection{write} -\textit{write} operation stores data in a sequential range of records. The input arguments to this -function are the location information (container, object, fork and starting record identifiers), local -buffer that stores the data to be written, number of records to be modified, conditional flags and a -user-specified version number. The range can be defined as the records with identifiers between starting -record identifier and the sum of starting record identifier and number of records to be modified. As an -example, if the starting record identifier is $1$ and the number of records to be modified is $3$; that -means records with identifiers $1, 2, 3$ will be included in the current write operation. Write operation -tries to store the given data into the given range of records by checking the given conditional flag and -by comparing the given version number with the existing version numbers in the user-specified range. - -One of the input arguments for the write operation is the conditional flag. The conditional flag can take a -combination of four different values. -\begin{itemize} -\item \textit{NONE}: Write should succeed without checking any version number of conditional flags. -\item \textit{ALL}: Write should only succeed if the user-specified version number is greater than all -the version numbers in the user-specified range. -\item \textit{UNTIL}: Write should \textit{continue} until it comes across a record that has a version -number greater than or equal to the user-specified version number. -\item \textit{AUTO}: In this case, user-specified version number can be ignored. The biggest version number -existing in the user-specified range is found and incremented; and the new data is written with this -incremented version number. -\end{itemize} - -A successful write operation returns the size of the written data and the newly assigned version number. -\subsubsection{read} -\textit{read} operation retrieves data from a sequential range of records. The input arguments to the read -operation are the location information (container, object, fork and starting record identifiers), a local -buffer to store the read data, number of records to be read, conditional flags and a user-specified version -number. The range in the read operation is identical to the range defined in the write operation. However, -the usage of the version number differs from the scenario in the write operation. In write operation, the -user specifies a version number that is \textit{assigned} to a range of records if conditional checks -are satisfied. However, in read operation the user specifies a version number so that it is -\textit{compared} against the existing version numbers in the specified range. At the end of a successful -read operation, records in the user-specified range will keep their original version numbers. - -One of the input arguments for the read operation is the conditional flag. The conditional flag can take a -combination of three different values. -\begin{itemize} -\item \textit{NONE}: Read should succeed without checking any version number of conditional flags. -\item \textit{ALL}: Read should only succeed if the user-specified version number is greater than all -the version numbers in the user-specified range. -\item \textit{UNTIL}: Read should \textit{continue} until it comes across a record that has a version -number greater than or equal to the user-specified version number. -\end{itemize} -A successful read operation returns the number of the records read in addition to the version number of -these records. - -\subsubsection{punch} -\textit{punch} removes data from the specified range of records. It behaves almost the same with the write -operation; it is basically a write with zero-length data. Punch have the same input and output arguments -with the write operation and it can only be applied to a range of records. Objects and containers do not -support punch operation. -\subsubsection{reset} -\textit{reset} returns an entity (container, object, fork or record) back to its -\textit{initial condition}. In an entity at its original condition, all the record(s) will have version -number \textit{zero} and will contain no data. Reset operation takes in the identifier information of the -entity to be reset as an input argument and it also supports conditional execution based on the existing -version number and given conditional flags. The conditional flags that can be used with the reset operation -are the same with the conditional flags used in read operation. A successful reset operation returns the -number of entities reset. - -\subsubsection{probe} -\textit{probe} returns information about a set of matching items. It can be called on the entire set of -available containers, objects or forks. Probe operation takes in the identifier information of the entity -(container, object or fork) to be probed as an input argument, an entity id to start with, a local -buffer to store the retrieved information and the maximum number of items for which the information will -be retrieved. The retrieved information consists of the identifier of the first -container/object/fork/record existing in the given entity, number of containers/objects/forks/records, -total number of records and the version numbers of these records. - -\section{Comparison with Existing Models} -Wisconson Storage System (WiSS) is a flexible storage system developed for the UNIX -environment~\cite{DBLP:journals/spe/ChouDKK85}. It supports unconventional types of data and it consists of -four distinct layers; \textit{level 0}, \textit{level 1}, -\textit{level 2} and \textit{level 3}. The basic structure units of WiSS are \textit{pages} at -\textit{level 0}. \textit{Level 0} performs I/O operations on pages. \textit{Level 1} is responsible for -maintaining a buffer pool of pages to make the access to the important pages faster. \textit{Level 2} of -WiSS carries some similarities to the ASG storage model. This level consists of \textit{records}, each of -which stored in an underlying page. A record consists of record type, record length and actual data. Record -length and actual data also exist in ASG records. Similar to the ASG storage model, each record is known by -a numerical identifier. Unlike ASG, records in WiSS have unique identifiers; whereas records in different -enclosing objects or forks etc. in ASG can have the same identifier. WiSS supports B-trees to index records -and long data items (a collection of pages) to store unlimited size of data; however in ASG there are no -such mechanisms yet. WiSS uses locking at the file granularity; whereas ASG supports versioning to provide -concurrency. \textit{Level 3} of WiSS has primitives to create or delete files using low-level primitives -which is similar to the Namespace Implementation we will discuss in Section~\ref{namespace}. - -\section{Sample Use Case: Namespace Implementation} -\label{namespace} -In this section we show how versioning and conditional operations can be used in ASG storage model; by implementing namespace operations as an example. - -A namespace can be defined as an abstract collection of unique keys. A key can be in a string -or integer format and it may appear in more than one namespaces with or without having the same -meaning; but it needs to be unique in each of these namespaces. The uniqueness property of -namespaces is analogous to the the uniqueness of each entry in a directory. Therefore it is -possible to implement directory operations with ASG storage model by layering a namespace -implementation atop it and associating ASG Storage entity identifiers with integer or string -keys. One important feature required by the traditional POSIX-style directories; but not -necessarily existent in namespace is the atomicity of operations. - -The namespace implementation has \textit{insert}, \textit{remove}, \textit{get}, \textit{list}, -\textit{update} and \textit{rename} operations. Each of these operations calls underlying ASG -storage model primitives, defined in Section \ref{primitives}, to accomplish their functionality. - -\begin{itemize} -\item \textit{Insert} associates new data with a given key in the namespace. There should not be any -existing data for the given key; otherwise \textit{insert} returns error. It calls underlying -\textit{write} primitive to insert the new data into the namespace. - -\item \textit{Remove} deletes a given key and its associated data from the namespace. The given key -should already exist in the namespace; otherwise \textit{remove} returns error. It calls underlying -\textit{read} and \textit{reset} primitives. - -\item \textit{Get} reads the data associated with a given key and it returns error if the given key -does not already exist in the namespace. Underlying \textit{read} functions is used by the \textit{get} -function without any conditional flags. - -\item \textit{List} returns information about a collection of keys in the namespace. This information -can be existing keys, key lenghts, data associated with existing keys or length of these data -segments depending on the function parameters. \textit{List} returns consistent information about -the existing keys in the namespace; meaning that it does not return information about keys that are -removed after its invocation and it returns updated information about keys that are modified after its -invocation. \textit{List} calls underlying \textit{probe} and \textit{read} functions to fetch information -about existing keys in a namespace. - -\item \textit{Rename} associates the data of an existing old key in a namespace with a new key in the same -namespace and deletes the old key from the namespace. Old key is removed from the namespace to prevent -duplicates. If the new key has already data associated with it, \textit{rename} overwrites that data. - \textit{Rename} calls underlying \textit{read} and \textit{write} operations. - -\item \textit{Update} modifies the data associated with a given key. If the given key does not exist in the -namespace, update returns error. It calls underlying \textit{read} and \textit{write} operations. - -\end{itemize} - -\section{Conclusions and Future Work} - -ASG storage model is good and can be further improved to support snapshotting, dynamic namespaces etc. - -%\section{This is Another Section} - -%Some embedded literal typset code might -%look like the following : - -%{\tt \small -%\begin{verbatim} -%int wrap_fact(ClientData clientData, -% Tcl_Interp *interp, -% int argc, char *argv[]) { -% int result; -% int arg0; -% if (argc != 2) { -% interp->result = "wrong # args"; -% return TCL_ERROR; -% } -% arg0 = atoi(argv[1]); -% result = fact(arg0); -% sprintf(interp->result,"%d",result); -% return TCL_OK; -%} -%\end{verbatim} -%} - -%Now we're going to cite somebody. Watch for the cite tag. -%Here it comes~\cite{Chaum1981,Diffie1976}. The tilde character (\~{}) -%in the source means a non-breaking space. This way, your reference will -%always be attached to the word that preceded it, instead of going to the -%next line. - -%\section{This Section has SubSections} -%\subsection{First SubSection} - -%Here's a typical figure reference. The figure is centered at the -%top of the column. It's scaled. It's explicitly placed. You'll -%have to tweak the numbers to get what you want.\\ - -% you can also use the wonderful epsfig package... -%\begin{figure}[t] -%\begin{center} -%\begin{picture}(300,150)(0,200) -%\put(-15,-30){\special{psfile = fig1.ps hscale = 50 vscale = 50}} -%\end{picture}\\ -%\end{center} -%\caption{Wonderful Flowchart} -%\end{figure} - -%This text came after the figure, so we'll casually refer to Figure 1 -%as we go on our merry way. - -%\subsection{New Subsection} - -%It can get tricky typesetting Tcl and C code in LaTeX because they share -%a lot of mystical feelings about certain magic characters. You -%will have to do a lot of escaping to typeset curly braces and percent -%signs, for example, like this: -%``The {\tt \%module} directive -%sets the name of the initialization function. This is optional, but is -%recommended if building a Tcl 7.5 module. -%Everything inside the {\tt \%\{, \%\}} -%block is copied directly into the output. allowing the inclusion of -%header files and additional C code." \\ - -%Sometimes you want to really call attention to a piece of text. You -%can center it in the column like this: -%\begin{center} -%{\tt \_1008e614\_Vector\_p} -%\end{center} -%and people will really notice it.\\ - -%\noindent -%The noindent at the start of this paragraph makes it clear that it's -%a continuation of the preceding text, not a new para in its own right. - - -%Now this is an ingenious way to get a forced space. -%{\tt Real~$*$} and {\tt double~$*$} are equivalent. - -%Now here is another way to call attention to a line of code, but instead -%of centering it, we noindent and bold it.\\ - -%\noindent -%{\bf \tt size\_t : fread ptr size nobj stream } \\ - -%And here we have made an indented para like a definition tag (dt) -%in HTML. You don't need a surrounding list macro pair. -%\begin{itemize} -%\item[] {\tt fread} reads from {\tt stream} into the array {\tt ptr} at -%most {\tt nobj} objects of size {\tt size}. {\tt fread} returns -%the number of objects read. -%\end{itemize} -%This concludes the definitions tag. - -%\subsection{How to Build Your Paper} - -%You have to run {\tt latex} once to prepare your references for -%munging. Then run {\tt bibtex} to build your bibliography metadata. -%Then run {\tt latex} twice to ensure all references have been resolved. -%If your source file is called {\tt usenixTemplate.tex} and your {\tt -% bibtex} file is called {\tt usenixTemplate.bib}, here's what you do: -%{\tt \small -%\begin{verbatim} -%latex usenixTemplate -%bibtex usenixTemplate -%latex usenixTemplate -%latex usenixTemplate -%\end{verbatim} -%} - - -%\subsection{Last SubSection} - -%Well, it's getting boring isn't it. This is the last subsection -%before we wrap it up. - -%\section{Acknowledgments} - -%A polite author always includes acknowledgments. Thank everyone, -%especially those who funded the work. - -%\section{Availability} - -%It's great when this section says that MyWonderfulApp is free software, -%available via anonymous FTP from - -%\begin{center} -%{\tt ftp.site.dom/pub/myname/Wonderful}\\ -%\end{center} - -%Also, it's even greater when you can write that information is also -%available on the Wonderful homepage at - -%\begin{center} -%{\tt http://www.site.dom/\~{}myname/SWIG} -%\end{center} - -%Now we get serious and fill in those references. Remember you will -%have to run latex twice on the document in order to resolve those -%cite tags you met earlier. This is where they get resolved. -%We've preserved some real ones in addition to the template-speak. -%After the bibliography you are DONE. - -{\footnotesize \bibliographystyle{acm} -\bibliography{pario,REFERENCES,paper}} - - -%\theendnotes - -\end{document} - - - - - - - - + +%% bare_conf.tex +%% V1.3 +%% 2007/01/11 +%% by Michael Shell +%% See: +%% http://www.michaelshell.org/ +%% for current contact information. +%% +%% This is a skeleton file demonstrating the use of IEEEtran.cls +%% (requires IEEEtran.cls version 1.7 or later) with an IEEE conference paper. +%% +%% Support sites: +%% http://www.michaelshell.org/tex/ieeetran/ +%% http://www.ctan.org/tex-archive/macros/latex/contrib/IEEEtran/ +%% and +%% http://www.ieee.org/ + +%%************************************************************************* +%% Legal Notice: +%% This code is offered as-is without any warranty either expressed or +%% implied; without even the implied warranty of MERCHANTABILITY or +%% FITNESS FOR A PARTICULAR PURPOSE! +%% User assumes all risk. +%% In no event shall IEEE or any contributor to this code be liable for +%% any damages or losses, including, but not limited to, incidental, +%% consequential, or any other damages, resulting from the use or misuse +%% of any information contained here. +%% +%% All comments are the opinions of their respective authors and are not +%% necessarily endorsed by the IEEE. +%% +%% This work is distributed under the LaTeX Project Public License (LPPL) +%% ( http://www.latex-project.org/ ) version 1.3, and may be freely used, +%% distributed and modified. A copy of the LPPL, version 1.3, is included +%% in the base LaTeX documentation of all distributions of LaTeX released +%% 2003/12/01 or later. +%% Retain all contribution notices and credits. +%% ** Modified files should be clearly indicated as such, including ** +%% ** renaming them and changing author support contact information. ** +%% +%% File list of work: IEEEtran.cls, IEEEtran_HOWTO.pdf, bare_adv.tex, +%% bare_conf.tex, bare_jrnl.tex, bare_jrnl_compsoc.tex +%%************************************************************************* + +% *** Authors should verify (and, if needed, correct) their LaTeX system *** +% *** with the testflow diagnostic prior to trusting their LaTeX platform *** +% *** with production work. IEEE's font choices can trigger bugs that do *** +% *** not appear when using other class files. *** +% The testflow support page is at: +% http://www.michaelshell.org/tex/testflow/ + + + +% Note that the a4paper option is mainly intended so that authors in +% countries using A4 can easily print to A4 and see how their papers will +% look in print - the typesetting of the document will not typically be +% affected with changes in paper size (but the bottom and side margins will). +% Use the testflow package mentioned above to verify correct handling of +% both paper sizes by the user's LaTeX system. +% +% Also note that the "draftcls" or "draftclsnofoot", not "draft", option +% should be used if it is desired that the figures are to be displayed in +% draft mode. +% +\documentclass[10pt, conference, compsocconf]{IEEEtran} +% Add the compsocconf option for Computer Society conferences. +% +% If IEEEtran.cls has not been installed into the LaTeX system files, +% manually specify the path to it like: +% \documentclass[conference]{../sty/IEEEtran} + + + + + +% Some very useful LaTeX packages include: +% (uncomment the ones you want to load) + + +% *** MISC UTILITY PACKAGES *** +% +%\usepackage{ifpdf} +% Heiko Oberdiek's ifpdf.sty is very useful if you need conditional +% compilation based on whether the output is pdf or dvi. +% usage: +% \ifpdf +% % pdf code +% \else +% % dvi code +% \fi +% The latest version of ifpdf.sty can be obtained from: +% http://www.ctan.org/tex-archive/macros/latex/contrib/oberdiek/ +% Also, note that IEEEtran.cls V1.7 and later provides a builtin +% \ifCLASSINFOpdf conditional that works the same way. +% When switching from latex to pdflatex and vice-versa, the compiler may +% have to be run twice to clear warning/error messages. + + + + + + +% *** CITATION PACKAGES *** +% +%\usepackage{cite} +% cite.sty was written by Donald Arseneau +% V1.6 and later of IEEEtran pre-defines the format of the cite.sty package +% \cite{} output to follow that of IEEE. Loading the cite package will +% result in citation numbers being automatically sorted and properly +% "compressed/ranged". e.g., [1], [9], [2], [7], [5], [6] without using +% cite.sty will become [1], [2], [5]--[7], [9] using cite.sty. cite.sty's +% \cite will automatically add leading space, if needed. Use cite.sty's +% noadjust option (cite.sty V3.8 and later) if you want to turn this off. +% cite.sty is already installed on most LaTeX systems. Be sure and use +% version 4.0 (2003-05-27) and later if using hyperref.sty. cite.sty does +% not currently provide for hyperlinked citations. +% The latest version can be obtained at: +% http://www.ctan.org/tex-archive/macros/latex/contrib/cite/ +% The documentation is contained in the cite.sty file itself. + + + + + + +% *** GRAPHICS RELATED PACKAGES *** +% +\ifCLASSINFOpdf + \usepackage[pdftex]{graphicx} + % declare the path(s) where your graphic files are + \graphicspath{{../pdf/}{../jpeg/}} + % and their extensions so you won't have to specify these with + % every instance of \includegraphics + \DeclareGraphicsExtensions{.pdf,.jpeg,.png} +\else + % or other class option (dvipsone, dvipdf, if not using dvips). graphicx + % will default to the driver specified in the system graphics.cfg if no + % driver is specified. + \usepackage[dvips]{graphicx} + % declare the path(s) where your graphic files are + \graphicspath{{../eps/}} + % and their extensions so you won't have to specify these with + % every instance of \includegraphics + \DeclareGraphicsExtensions{.eps} +\fi +% graphicx was written by David Carlisle and Sebastian Rahtz. It is +% required if you want graphics, photos, etc. graphicx.sty is already +% installed on most LaTeX systems. The latest version and documentation can +% be obtained at: +% http://www.ctan.org/tex-archive/macros/latex/required/graphics/ +% Another good source of documentation is "Using Imported Graphics in +% LaTeX2e" by Keith Reckdahl which can be found as epslatex.ps or +% epslatex.pdf at: http://www.ctan.org/tex-archive/info/ +% +% latex, and pdflatex in dvi mode, support graphics in encapsulated +% postscript (.eps) format. pdflatex in pdf mode supports graphics +% in .pdf, .jpeg, .png and .mps (metapost) formats. Users should ensure +% that all non-photo figures use a vector format (.eps, .pdf, .mps) and +% not a bitmapped formats (.jpeg, .png). IEEE frowns on bitmapped formats +% which can result in "jaggedy"/blurry rendering of lines and letters as +% well as large increases in file sizes. +% +% You can find documentation about the pdfTeX application at: +% http://www.tug.org/applications/pdftex + + + + + +% *** MATH PACKAGES *** +% +%\usepackage[cmex10]{amsmath} +% A popular package from the American Mathematical Society that provides +% many useful and powerful commands for dealing with mathematics. If using +% it, be sure to load this package with the cmex10 option to ensure that +% only type 1 fonts will utilized at all point sizes. Without this option, +% it is possible that some math symbols, particularly those within +% footnotes, will be rendered in bitmap form which will result in a +% document that can not be IEEE Xplore compliant! +% +% Also, note that the amsmath package sets \interdisplaylinepenalty to 10000 +% thus preventing page breaks from occurring within multiline equations. Use: +%\interdisplaylinepenalty=2500 +% after loading amsmath to restore such page breaks as IEEEtran.cls normally +% does. amsmath.sty is already installed on most LaTeX systems. The latest +% version and documentation can be obtained at: +% http://www.ctan.org/tex-archive/macros/latex/required/amslatex/math/ + + + + + +% *** SPECIALIZED LIST PACKAGES *** +% +%\usepackage{algorithmic} +% algorithmic.sty was written by Peter Williams and Rogerio Brito. +% This package provides an algorithmic environment fo describing algorithms. +% You can use the algorithmic environment in-text or within a figure +% environment to provide for a floating algorithm. Do NOT use the algorithm +% floating environment provided by algorithm.sty (by the same authors) or +% algorithm2e.sty (by Christophe Fiorio) as IEEE does not use dedicated +% algorithm float types and packages that provide these will not provide +% correct IEEE style captions. The latest version and documentation of +% algorithmic.sty can be obtained at: +% http://www.ctan.org/tex-archive/macros/latex/contrib/algorithms/ +% There is also a support site at: +% http://algorithms.berlios.de/index.html +% Also of interest may be the (relatively newer and more customizable) +% algorithmicx.sty package by Szasz Janos: +% http://www.ctan.org/tex-archive/macros/latex/contrib/algorithmicx/ + + + + +% *** ALIGNMENT PACKAGES *** +% +%\usepackage{array} +% Frank Mittelbach's and David Carlisle's array.sty patches and improves +% the standard LaTeX2e array and tabular environments to provide better +% appearance and additional user controls. As the default LaTeX2e table +% generation code is lacking to the point of almost being broken with +% respect to the quality of the end results, all users are strongly +% advised to use an enhanced (at the very least that provided by array.sty) +% set of table tools. array.sty is already installed on most systems. The +% latest version and documentation can be obtained at: +% http://www.ctan.org/tex-archive/macros/latex/required/tools/ + + +%\usepackage{mdwmath} +%\usepackage{mdwtab} +% Also highly recommended is Mark Wooding's extremely powerful MDW tools, +% especially mdwmath.sty and mdwtab.sty which are used to format equations +% and tables, respectively. The MDWtools set is already installed on most +% LaTeX systems. The lastest version and documentation is available at: +% http://www.ctan.org/tex-archive/macros/latex/contrib/mdwtools/ + + +% IEEEtran contains the IEEEeqnarray family of commands that can be used to +% generate multiline equations as well as matrices, tables, etc., of high +% quality. + + +%\usepackage{eqparbox} +% Also of notable interest is Scott Pakin's eqparbox package for creating +% (automatically sized) equal width boxes - aka "natural width parboxes". +% Available at: +% http://www.ctan.org/tex-archive/macros/latex/contrib/eqparbox/ + + + + + +% *** SUBFIGURE PACKAGES *** +%\usepackage[tight,footnotesize]{subfigure} +% subfigure.sty was written by Steven Douglas Cochran. This package makes it +% easy to put subfigures in your figures. e.g., "Figure 1a and 1b". For IEEE +% work, it is a good idea to load it with the tight package option to reduce +% the amount of white space around the subfigures. subfigure.sty is already +% installed on most LaTeX systems. The latest version and documentation can +% be obtained at: +% http://www.ctan.org/tex-archive/obsolete/macros/latex/contrib/subfigure/ +% subfigure.sty has been superceeded by subfig.sty. + + + +%\usepackage[caption=false]{caption} +%\usepackage[font=footnotesize]{subfig} +% subfig.sty, also written by Steven Douglas Cochran, is the modern +% replacement for subfigure.sty. However, subfig.sty requires and +% automatically loads Axel Sommerfeldt's caption.sty which will override +% IEEEtran.cls handling of captions and this will result in nonIEEE style +% figure/table captions. To prevent this problem, be sure and preload +% caption.sty with its "caption=false" package option. This is will preserve +% IEEEtran.cls handing of captions. Version 1.3 (2005/06/28) and later +% (recommended due to many improvements over 1.2) of subfig.sty supports +% the caption=false option directly: +%\usepackage[caption=false,font=footnotesize]{subfig} +% +% The latest version and documentation can be obtained at: +% http://www.ctan.org/tex-archive/macros/latex/contrib/subfig/ +% The latest version and documentation of caption.sty can be obtained at: +% http://www.ctan.org/tex-archive/macros/latex/contrib/caption/ + + + + +% *** FLOAT PACKAGES *** +% +%\usepackage{fixltx2e} +% fixltx2e, the successor to the earlier fix2col.sty, was written by +% Frank Mittelbach and David Carlisle. This package corrects a few problems +% in the LaTeX2e kernel, the most notable of which is that in current +% LaTeX2e releases, the ordering of single and double column floats is not +% guaranteed to be preserved. Thus, an unpatched LaTeX2e can allow a +% single column figure to be placed prior to an earlier double column +% figure. The latest version and documentation can be found at: +% http://www.ctan.org/tex-archive/macros/latex/base/ + + + +%\usepackage{stfloats} +% stfloats.sty was written by Sigitas Tolusis. This package gives LaTeX2e +% the ability to do double column floats at the bottom of the page as well +% as the top. (e.g., "\begin{figure*}[!b]" is not normally possible in +% LaTeX2e). It also provides a command: +%\fnbelowfloat +% to enable the placement of footnotes below bottom floats (the standard +% LaTeX2e kernel puts them above bottom floats). This is an invasive package +% which rewrites many portions of the LaTeX2e float routines. It may not work +% with other packages that modify the LaTeX2e float routines. The latest +% version and documentation can be obtained at: +% http://www.ctan.org/tex-archive/macros/latex/contrib/sttools/ +% Documentation is contained in the stfloats.sty comments as well as in the +% presfull.pdf file. Do not use the stfloats baselinefloat ability as IEEE +% does not allow \baselineskip to stretch. Authors submitting work to the +% IEEE should note that IEEE rarely uses double column equations and +% that authors should try to avoid such use. Do not be tempted to use the +% cuted.sty or midfloat.sty packages (also by Sigitas Tolusis) as IEEE does +% not format its papers in such ways. + + + + + +% *** PDF, URL AND HYPERLINK PACKAGES *** +% +%\usepackage{url} +% url.sty was written by Donald Arseneau. It provides better support for +% handling and breaking URLs. url.sty is already installed on most LaTeX +% systems. The latest version can be obtained at: +% http://www.ctan.org/tex-archive/macros/latex/contrib/misc/ +% Read the url.sty source comments for usage information. Basically, +% \url{my_url_here}. + + + + + +% *** Do not adjust lengths that control margins, column widths, etc. *** +% *** Do not use packages that alter fonts (such as pslatex). *** +% There should be no need to do such things with IEEEtran.cls V1.6 and later. +% (Unless specifically asked to do so by the journal or conference you plan +% to submit to, of course. ) + + +% correct bad hyphenation here +\hyphenation{op-tical net-works semi-conduc-tor} + + +\begin{document} +% +% paper title +% can use linebreaks \\ within to get better formatting as desired +\title{Concept and Primitives of the ASG Object-Based Storage System} + + +% author names and affiliations +% use a multiple column layout for up to two different +% affiliations + +%\author{\IEEEauthorblockN{Authors Name/s per 1st Affiliation (Author)} +%\IEEEauthorblockA{line 1 (of Affiliation): dept. name of organization\\ +%line 2: name of organization, acronyms acceptable\\ +%line 3: City, Country\\ +%line 4: Email: name(a)xyz.com} +%\and +%\IEEEauthorblockN{Authors Name/s per 2nd Affiliation (Author)} +%\IEEEauthorblockA{line 1 (of Affiliation): dept. name of organization\\ +%line 2: name of organization, acronyms acceptable\\ +%line 3: City, Country\\ +%line 4: Email: name(a)xyz.com} +%} + +% conference papers do not typically use \thanks and this command +% is locked out in conference mode. If really needed, such as for +% the acknowledgment of grants, issue a \IEEEoverridecommandlockouts +% after \documentclass + +% for over three affiliations, or if they all won't fit within the width +% of the page, use this alternative format: +% +%\author{\IEEEauthorblockN{Michael Shell\IEEEauthorrefmark{1}, +%Homer Simpson\IEEEauthorrefmark{2}, +%James Kirk\IEEEauthorrefmark{3}, +%Montgomery Scott\IEEEauthorrefmark{3} and +%Eldon Tyrell\IEEEauthorrefmark{4}} +%\IEEEauthorblockA{\IEEEauthorrefmark{1}School of Electrical and Computer Engineering\\ +%Georgia Institute of Technology, +%Atlanta, Georgia 30332--0250\\ Email: see http://www.michaelshell.org/contact.html} +%\IEEEauthorblockA{\IEEEauthorrefmark{2}Twentieth Century Fox, Springfield, USA\\ +%Email: homer(a)thesimpsons.com} +%\IEEEauthorblockA{\IEEEauthorrefmark{3}Starfleet Academy, San Francisco, California 96678-2391\\ +%Telephone: (800) 555--1212, Fax: (888) 555--1212} +%\IEEEauthorblockA{\IEEEauthorrefmark{4}Tyrell Inc., 123 Replicant Street, Los Angeles, California 90210--4321}} + + + + +% use for special paper notices +%\IEEEspecialpapernotice{(Invited Paper)} + + + + +% make the title area +\maketitle + + +\begin{abstract} +Distributed object-based storage models are an increasingly popular alternative +to traditional block-based or file-based storage abstractions in large-scale +storage systems. Object-based storage models store and access data +in discrete, byte-addressable containers to simplify data management, to provide +easier data sharing, locality and better security compared to traditional storage abstractions. +Although many large-scale storage systems share common goals of performance, scalability +and fault tolerance, their underlying storage models are typically tailored +to specific classes of applications through incremental evolution of previous standards. +In this paper, we propose a new distributed object-based storage API +that seeks to combine data access features (i.e. concurrent reads\&writes, versioning, +conditional operations, failure handling) into a unified API that is suitable for use as a +base model in a wide range of contexts, including file systems, cloud storage, databases, HPC +systems and big data. We identify the common requirements of these data models, identify key +features of previous object storage interfaces, and explore how they can be unified effectively +without sacrificing usability. We validate the effectiveness of our approach with case studies +that evaluate how the storage API we are proposing can be used to construct a namespace and a +column-store database. +\end{abstract} + +%\begin{IEEEkeywords} +%component; formatting; style; styling; + +%\end{IEEEkeywords} + + +% For peer review papers, you can put extra information on the cover +% page as needed: +% \ifCLASSOPTIONpeerreview +% \begin{center} \bfseries EDICS Category: 3-BBND \end{center} +% \fi +% +% For peerreview papers, this IEEEtran command inserts a page break and +% creates the second title. It will be ignored for other modes. +\IEEEpeerreviewmaketitle + + + +\section{Introduction} + +Storage devices have seen a lot of improvement in terms of performance and complexity in the last decades; +but they could not catch the development rate of the computation and processing units of the computers. +Modern applications demand high-performance, security and portability from the storage systems of the +computers and this requires the storage units to redefine the abstraction to store and manage the data. +In order to meet these demands, a new storage model called \textit{object-based storage} has been developed +~\cite{1222722, osd-3}. Storage devices that store and access the data according to the object-based storage +model are called as \textit{object-based storage devices}. Storage system developers prefer to build software +implementations of object-based storage model on block-based devices; since there is no hardware +implementation of object-based storage devices yet. + +An \textit{object} is an ordered logical collection of bytes with a numerical identifier. Objects consist +of data, user settable attributes and device managed metadata including security policies that protect data +~\cite{1222722, Welch08scalableperformance}. They have variable sizes and they can be used to store any +kind of data in the storage system. Having data organized as objects makes it possible for the +disk to do data management operations on behalf of the user ~\cite{5944457} and offloads the storage +management from the host operating system to the object-based storage. User-settable attributes can +be further enriched to set performance goals, manage replicas, or handle concurrency in object-based +storage model to enable self-management capabilities and they give the user an effective control of data +on the disk~\cite{gibson:storage, 5944457}. Object-based storage is also suitable for realizing +\textit{active storage}~\cite{5944457,Acharya:1998:ADP:291006.291026, riedel:thesis}; because objects +provide more features to encapsulate the computations compared to traditional blocks. + +Although object-based storage model has been investigated in many research studies and several +optimizations have been introduced for this model, we believe that the object-based storage model +primitives need to be revisited. Transactional Object Storage Device +(TOSD) work~\cite{carns2010object} shows that object-based storage is a common component of many parallel +file systems and it introduces three optimizations to the object-based storage model in order to serve +highly concurrent workloads better; atomicity, versioning and commutativity. Goodell et al. +~\cite{goodell-pdsw12} extended the POSIX API by organizing the storage around data objects in order to map +complex data structures to these data objects and have direct access between the data objects and +applications. Carns et al. ~\cite{carns-pdsw12} investigated conditional update operations as an +alternative to distributed pessimistic locking operations in object-based storage systems. + +We believe an object-based storage system can offer a lot more sophisticated functionalities by +implementing versioning and conditional operations and we introduce a new object-based storage +system model, ASG, that unifies the conditional operation and versioning features previously explored in +related studies described in the previous paragraph. The rest of the paper is organized as follows; +Section 2 talks about related work in object-based storage systems. Section 3 describes the ASG +architecture in details. Section 4 compares the ASG model with existing T10 model and identifies the +advantages\&disadvantages of each model. We give a sample usage scenario of ASG model in Section 5 followed +by the conclusion and possible directions for future development of ASG in Section 6. + +\section{Related Work} + +Network-Attached Secure Disk (NASD) ~\cite{gibson:storage} from Carnegie Mellon is the primary study on +object-based storage systems and it led to specifications of standards~\cite{osd-3} for object-based +storage devices. It is based on the idea of integrating the disk management functions into the storage +device and it offers a variable-length object-storage interface. NASD introduces variable-length objects +with attributes, rather than fixed-length traditional blocks, to enable self-management and to obviate the +need to know about the host operating system. Moving data management to the storage disks increases the +networking, security and space management capabilities. + +OSD+~\cite{springerlink:10.1007/s10766-012-0207-8,10.1109/SBAC-PAD.2011.12} presents a model very similar to +the one specified by the OSD standard ~\cite{osd-3} except for the addition of dedicated directory objects. +The directory objects in OSD+ store file names and attributes and support metadata related operations. Each +OSD+ stores a single directory object. In order to guarantee correctness and atomicity, OSD+ relies on the +back-end file system (Ext3, Ext4) and three-phase commit protocol~\cite{Skeen:1983:FMC:1313337.1313750}. + +Panasas File System~\cite{Welch08scalableperformance} is built on object-based storage devices. Panasas +File System clients have direct access to the objects which are a combination of data and attributes. OSD +wire protocol of Panasas uses the operations from the OSD standard~\cite{osd-3} to enable byte-oriented +access to data, to manipulate attributes and to create or delete objects. Panasas clients use iSCSI/OSD +protocol to access the object-based storage devices. + +Lustre ~\cite{lustre-web} is a distributed file system based on object-based storage. Lustre has two types +of servers; Metadata Server (MDS) and Object Storage Server (OSS). Actual metadata is stored on Metadata +Targets (MDTs) and file data is stored on Object Storage Targets (OSTs) as objects. MDS is responsible +for handling metadata requests and making the metadata stored in MDTs available to the clients; whereas +OSS is responsible for filesystem I/O and for providing access to file data stored on OSTs. + +\textit{Ursa Minor} is a parallel file system supporting write versioning; however, \textit{Ursa Minor} keeps +the existing object-storage interface~\cite{osd-3} mostly intact except for introducing \textit{slices} which +are fragments of object data and it uses timestamps to distinguish different versions of data. + +Other related work~\cite{Devulapalli07integratingparallel} from Ohio Supercomputing Center looked at +mapping Parallel Virtual File System~\cite{pvfs-web} on top of an existing object-based storage +emulation~\cite{1306916}. This mapping moved the functionality of the common components of a traditional +storage system; such as I/O, directory or metadata servers to OSDs and improved the performance +of the overall system due to the capabilities of the object-based storage devices.~\cite{OSDCluster08, OSDPDSI08, 10.1109/SNAPI.2008.14} + +\section{Implementation Strategies} +\label{primitives} +In this section we describe the ASG storage model, its fundamental building blocks and basic primitives. + +\subsection{Architecture} +The main architecture of ASG storage model is shown in Figure \ref{architecture}. + +\begin{figure}[h] +\centering +\includegraphics[width=\columnwidth, keepaspectratio]{overview.pdf} +\caption{Architecture of ASG Storage Model} +\label{architecture} +\end{figure} + +\begin{itemize} +\item The basic building block of the ASG storage model is a \textit{record}. Each record includes a key, a +version number, data and length of data. The key, version number and length of data are represented with +integers; whereas as data is a variable length array of bytes. Key is the numerical identifier of a record. +Version numbers are used to order writes to the record. Data field can be empty and a record at its +\textit{initial condition} will have version number \textit{zero} and will contain no data. +\item Fork is a collection of records and it provides a distinct namespace for the records it contains. Each +fork is identified by a fixed-length integer. +\item Object is a collection of forks and it provides a distinct namespace for the forks it contains. Each +object is identified by a fixed-length integer. +\item Container is a collection of objects and it provides a distinct namespace for the objects it contains. +Each container is identified by a fixed-length integer. Containers partition the storage system into logical +units; as an example each container could contain a distinct file system. +\end{itemize} + +The record, fork and object identifiers in ASG storage model are not global. For example; two different +containers can have objects with the same identifiers. Similarly, two different objects/forks can have +forks/records with the same identifiers. + +\subsection{Operations} +In this section we describe the ASG storage model operations a client use to interact with the storage +system. + +\subsubsection{write} +\textit{write} operation stores data in a sequential range of records. The input arguments to this +function are the location information (container, object, fork and starting record identifiers), local +buffer that stores the data to be written, number of records to be modified, conditional flags and a +user-specified version number. The range can be defined as the records with identifiers between starting +record identifier and the sum of starting record identifier and number of records to be modified. As an +example, if the starting record identifier is $1$ and the number of records to be modified is $3$; that +means records with identifiers $1, 2, 3$ will be included in the current write operation. Write operation +tries to store the given data into the given range of records by checking the given conditional flag and +by comparing the given version number with the existing version numbers in the user-specified range. + +One of the input arguments for the write operation is the conditional flag. The conditional flag can take a +combination of four different values. +\begin{itemize} +\item \textit{NONE}: Write should succeed without checking any version number of conditional flags. +\item \textit{ALL}: Write should only succeed if the user-specified version number is greater than all +the version numbers in the user-specified range. +\item \textit{UNTIL}: Write should \textit{continue} until it comes across a record that has a version +number greater than or equal to the user-specified version number. +\item \textit{AUTO}: In this case, user-specified version number can be ignored. The biggest version number +existing in the user-specified range is found and incremented; and the new data is written with this +incremented version number. +\end{itemize} + +A successful write operation returns the size of the written data and the newly assigned version number. +\subsubsection{read} +\textit{read} operation retrieves data from a sequential range of records. The input arguments to the read +operation are the location information (container, object, fork and starting record identifiers), a local +buffer to store the read data, number of records to be read, conditional flags and a user-specified version +number. The range in the read operation is identical to the range defined in the write operation. However, +the usage of the version number differs from the scenario in the write operation. In write operation, the +user specifies a version number that is \textit{assigned} to a range of records if conditional checks +are satisfied. However, in read operation the user specifies a version number so that it is +\textit{compared} against the existing version numbers in the specified range. At the end of a successful +read operation, records in the user-specified range will keep their original version numbers. + +One of the input arguments for the read operation is the conditional flag. The conditional flag can take a +combination of three different values. +\begin{itemize} +\item \textit{NONE}: Read should succeed without checking any version number of conditional flags. +\item \textit{ALL}: Read should only succeed if the user-specified version number is greater than all +the version numbers in the user-specified range. +\item \textit{UNTIL}: Read should \textit{continue} until it comes across a record that has a version +number greater than or equal to the user-specified version number. +\end{itemize} +A successful read operation returns the number of the records read in addition to the version number of +these records. + +\subsubsection{punch} +\textit{punch} removes data from the specified range of records. It behaves almost the same with the write +operation; it is basically a write with zero-length data. Punch have the same input and output arguments +with the write operation and it can only be applied to a range of records. Objects and containers do not +support punch operation. +\subsubsection{reset} +\textit{reset} returns an entity (container, object, fork or record) back to its +\textit{initial condition}. In an entity at its original condition, all the record(s) will have version +number \textit{zero} and will contain no data. Reset operation takes in the identifier information of the +entity to be reset as an input argument and it also supports conditional execution based on the existing +version number and given conditional flags. The conditional flags that can be used with the reset operation +are the same with the conditional flags used in read operation. A successful reset operation returns the +number of entities reset. + +\subsubsection{probe} +\textit{probe} returns information about a set of matching items. It can be called on the entire set of +available containers, objects or forks. Probe operation takes in the identifier information of the entity +(container, object or fork) to be probed as an input argument, an entity id to start with, a local +buffer to store the retrieved information and the maximum number of items for which the information will +be retrieved. The retrieved information consists of the identifier of the first +container/object/fork/record existing in the given entity, number of containers/objects/forks/records, +total number of records and the version numbers of these records. + +\section{Comparison with Existing Models} +Wisconson Storage System (WiSS) is a flexible storage system developed for the UNIX +environment~\cite{DBLP:journals/spe/ChouDKK85}. It supports unconventional types of data and it consists of +four distinct layers; \textit{level 0}, \textit{level 1}, +\textit{level 2} and \textit{level 3}. The basic structure units of WiSS are \textit{pages} at +\textit{level 0}. \textit{Level 0} performs I/O operations on pages. \textit{Level 1} is responsible for +maintaining a buffer pool of pages to make the access to the important pages faster. \textit{Level 2} of +WiSS carries some similarities to the ASG storage model. This level consists of \textit{records}, each of +which stored in an underlying page. A record consists of record type, record length and actual data. Record +length and actual data also exist in ASG records. Similar to the ASG storage model, each record is known by +a numerical identifier. Unlike ASG, records in WiSS have unique identifiers; whereas records in different +enclosing objects or forks etc. in ASG can have the same identifier. WiSS supports B-trees to index records +and long data items (a collection of pages) to store unlimited size of data; however in ASG there are no +such mechanisms yet. WiSS uses locking at the file granularity; whereas ASG supports versioning to provide +concurrency. \textit{Level 3} of WiSS has primitives to create or delete files using low-level primitives +which is similar to the Namespace Implementation we will discuss in Section~\ref{namespace}. + +\section{Sample Use Case: Namespace Implementation} +\label{namespace} +In this section we show how versioning and conditional operations can be used in ASG storage model; by implementing namespace operations as an example. + +A namespace can be defined as an abstract collection of unique keys. A key can be in a string +or integer format and it may appear in more than one namespaces with or without having the same +meaning; but it needs to be unique in each of these namespaces. The uniqueness property of +namespaces is analogous to the the uniqueness of each entry in a directory. Therefore it is +possible to implement directory operations with ASG storage model by layering a namespace +implementation atop it and associating ASG Storage entity identifiers with integer or string +keys. One important feature required by the traditional POSIX-style directories; but not +necessarily existent in namespace is the atomicity of operations. + +The namespace implementation has \textit{insert}, \textit{remove}, \textit{get}, \textit{list}, +\textit{update} and \textit{rename} operations. Each of these operations calls underlying ASG +storage model primitives, defined in Section \ref{primitives}, to accomplish their functionality. + +\begin{itemize} +\item \textit{Insert} associates new data with a given key in the namespace. There should not be any +existing data for the given key; otherwise \textit{insert} returns error. It calls underlying +\textit{write} primitive to insert the new data into the namespace. + +\item \textit{Remove} deletes a given key and its associated data from the namespace. The given key +should already exist in the namespace; otherwise \textit{remove} returns error. It calls underlying +\textit{read} and \textit{reset} primitives. + +\item \textit{Get} reads the data associated with a given key and it returns error if the given key +does not already exist in the namespace. Underlying \textit{read} functions is used by the \textit{get} +function without any conditional flags. + +\item \textit{List} returns information about a collection of keys in the namespace. This information +can be existing keys, key lenghts, data associated with existing keys or length of these data +segments depending on the function parameters. \textit{List} returns consistent information about +the existing keys in the namespace; meaning that it does not return information about keys that are +removed after its invocation and it returns updated information about keys that are modified after its +invocation. \textit{List} calls underlying \textit{probe} and \textit{read} functions to fetch information +about existing keys in a namespace. + +\item \textit{Rename} associates the data of an existing old key in a namespace with a new key in the same +namespace and deletes the old key from the namespace. Old key is removed from the namespace to prevent +duplicates. If the new key has already data associated with it, \textit{rename} overwrites that data. + \textit{Rename} calls underlying \textit{read} and \textit{write} operations. + +\item \textit{Update} modifies the data associated with a given key. If the given key does not exist in the +namespace, update returns error. It calls underlying \textit{read} and \textit{write} operations. + +\end{itemize} + +% no \IEEEPARstart +%This demo file is intended to serve as a ``starter file'' +%for IEEE conference papers produced under \LaTeX\ using +%IEEEtran.cls version 1.7 and later. + +%All manuscripts must be in English. These guidelines include complete descriptions of the fonts, spacing, and related information for producing your proceedings manuscripts. Please follow them and if you have any questions, direct them to the production editor in charge of your proceedings at Conference Publishing Services (CPS): Phone +1 (714) 821-8380 or Fax +1 (714) 761-1784. +% You must have at least 2 lines in the paragraph with the drop letter +% (should never be an issue) + +%\subsection{Subsection Heading Here} +%Subsection text here. + + +%\subsubsection{Subsubsection Heading Here} +%Subsubsection text here. + +%\section{Type style and Fonts} +%Wherever Times is specified, Times Roman or Times New Roman may be used. If neither is available on your system, please use the font closest in appearance to Times. Avoid using bit-mapped fonts if possible. True-Type 1 or Open Type fonts are preferred. Please embed symbol fonts, as well, for math, etc. + + +% An example of a floating figure using the graphicx package. +% Note that \label must occur AFTER (or within) \caption. +% For figures, \caption should occur after the \includegraphics. +% Note that IEEEtran v1.7 and later has special internal code that +% is designed to preserve the operation of \label within \caption +% even when the captionsoff option is in effect. However, because +% of issues like this, it may be the safest practice to put all your +% \label just after \caption rather than within \caption{}. +% +% Reminder: the "draftcls" or "draftclsnofoot", not "draft", class +% option should be used if it is desired that the figures are to be +% displayed while in draft mode. +% +%\begin{figure}[!t] +%\centering +%\includegraphics[width=2.5in]{myfigure} +% where an .eps filename suffix will be assumed under latex, +% and a .pdf suffix will be assumed for pdflatex; or what has been declared +% via \DeclareGraphicsExtensions. +%\caption{Simulation Results} +%\label{fig_sim} +%\end{figure} + +% Note that IEEE typically puts floats only at the top, even when this +% results in a large percentage of a column being occupied by floats. + + +% An example of a double column floating figure using two subfigures. +% (The subfig.sty package must be loaded for this to work.) +% The subfigure \label commands are set within each subfloat command, the +% \label for the overall figure must come after \caption. +% \hfil must be used as a separator to get equal spacing. +% The subfigure.sty package works much the same way, except \subfigure is +% used instead of \subfloat. +% +%\begin{figure*}[!t] +%\centerline{\subfloat[Case I]\includegraphics[width=2.5in]{subfigcase1}% +%\label{fig_first_case}} +%\hfil +%\subfloat[Case II]{\includegraphics[width=2.5in]{subfigcase2}% +%\label{fig_second_case}}} +%\caption{Simulation results} +%\label{fig_sim} +%\end{figure*} +% +% Note that often IEEE papers with subfigures do not employ subfigure +% captions (using the optional argument to \subfloat), but instead will +% reference/describe all of them (a), (b), etc., within the main caption. + + +% An example of a floating table. Note that, for IEEE style tables, the +% \caption command should come BEFORE the table. Table text will default to +% \footnotesize as IEEE normally uses this smaller font for tables. +% The \label must come after \caption as always. +% +%\begin{table}[!t] +%% increase table row spacing, adjust to taste +%\renewcommand{\arraystretch}{1.3} +% if using array.sty, it might be a good idea to tweak the value of +% \extrarowheight as needed to properly center the text within the cells +%\caption{An Example of a Table} +%\label{table_example} +%\centering +%% Some packages, such as MDW tools, offer better commands for making tables +%% than the plain LaTeX2e tabular which is used here. +%\begin{tabular}{|c||c|} +%\hline +%One & Two\\ +%\hline +%Three & Four\\ +%\hline +%\end{tabular} +%\end{table} + + +% Note that IEEE does not put floats in the very first column - or typically +% anywhere on the first page for that matter. Also, in-text middle ("here") +% positioning is not used. Most IEEE journals/conferences use top floats +% exclusively. Note that, LaTeX2e, unlike IEEE journals/conferences, places +% footnotes above bottom floats. This can be corrected via the \fnbelowfloat +% command of the stfloats package. + + + +\section{Conclusions and Future Work} +ASG storage model is good and can be further improved to support snapshotting, dynamic namespaces etc. + +% conference papers do not normally have an appendix + + +% use section* for acknowledgement +%\section*{Acknowledgment} + + +%The authors would like to thank... +%more thanks here + + +% trigger a \newpage just before the given reference +% number - used to balance the columns on the last page +% adjust value as needed - may need to be readjusted if +% the document is modified later +%\IEEEtriggeratref{8} +% The "triggered" command can be changed if desired: +%\IEEEtriggercmd{\enlargethispage{-5in}} + +% references section + +% can use a bibliography generated by BibTeX as a .bbl file +% BibTeX documentation can be easily obtained at: +% http://www.ctan.org/tex-archive/biblio/bibtex/contrib/doc/ +% The IEEEtran BibTeX style support page is at: +% http://www.michaelshell.org/tex/ieeetran/bibtex/ +%\bibliographystyle{IEEEtran} +% argument is your BibTeX string definitions and bibliography database(s) +%\bibliography{IEEEabrv,../bib/paper} +% +% <OR> manually copy in the resultant .bbl file +% set second argument of \begin to the number of references +% (used to reserve space for the reference number labels box) +%\begin{thebibliography}{1} + +%\bibitem{IEEEhowto:kopka} +%H.~Kopka and P.~W. Daly, \emph{A Guide to \LaTeX}, 3rd~ed.\hskip 1em plus +% 0.5em minus 0.4em\relax Harlow, England: Addison-Wesley, 1999. + +%\end{thebibliography} + +\bibliographystyle{plain} +\bibliography{pario,REFERENCES,paper} + +% that's all folks +\end{document} + + hooks/post-receive --

1 0

C-Utils Repository branch, cleanup, updated. 020053b2b05370082d43aa924d39bba828286132
by noreply＠mcs.anl.gov 28 Mar '13

28 Mar '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "C-Utils Repository". The branch, cleanup has been updated via 020053b2b05370082d43aa924d39bba828286132 (commit) from 7e9cfef4ee5ecf2587ece0d8946fcd69b384cbee (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 020053b2b05370082d43aa924d39bba828286132 Author: Dries Kimpe <dkimpe(a)mcs.anl.gov> Date: Thu Mar 28 11:42:06 2013 -0500 Only bundle libev when --enable-libev is specified ----------------------------------------------------------------------- Summary of changes: configure.ac | 33 ++++++++++++++++++++++++++++++--- libev/Makefile.subdir | 2 ++ 2 files changed, 32 insertions(+), 3 deletions(-) Diff of changes: diff --git a/configure.ac b/configure.ac index 6568b6b..c996202 100644 --- a/configure.ac +++ b/configure.ac @@ -25,6 +25,36 @@ dnl AC_REQUIRE_AUX_FILE([tap-driver.sh]) AC_CHECK_SIZEOF([long int]) + +dnl ====================================================================== +dnl bundle libev? +dnl ====================================================================== + +AC_ARG_ENABLE(libev, + [AS_HELP_STRING( + [--enable-libev], + [Bundle libev into c-utils]) + ]) +BUNDLE_LIBEV= +AS_IF([test "x$enable_libev" = "xyes"], [BUNDLE_LIBEV=1]) +AC_SUBST([BUNDLE_LIBEV]) + +AM_CONDITIONAL([LIBEV],[test "x$BUNDLE_LIBEV" == "x1"]) + +AC_MSG_CHECKING([if we need to bundle libev]) + +if test "x$BUNDLE_LIBEV" == "x1"; then + AC_MSG_RESULT([yes]) + + # bring in libev tests + m4_include([m4/libev.m4]) + +else + AC_MSG_RESULT([no]) +fi + + + dnl ====================================================================== dnl Try harder to be valgrind safe dnl ====================================================================== @@ -133,9 +163,6 @@ AC_ARG_ENABLE(coverage, ) AC_SUBST(ENABLE_COVERAGE) -# bring in libev tests -m4_include([m4/libev.m4]) - AC_CONFIG_FILES([Makefile]) AC_CONFIG_LINKS([config.h:c-utils-config.h]) diff --git a/libev/Makefile.subdir b/libev/Makefile.subdir index 7a2b9f0..58c8e6b 100644 --- a/libev/Makefile.subdir +++ b/libev/Makefile.subdir @@ -1,4 +1,5 @@ +if LIBEV # build libev as convenience library so we can easily override the CFLAGS/etc. # for this library noinst_LTLIBRARIES += libev/libev.la @@ -29,3 +30,4 @@ noinst_HEADERS += \ public_headers += \ libev/ev.h +endif hooks/post-receive -- C-Utils Repository

1 0

C-Utils Repository branch, cleanup, updated. 7e9cfef4ee5ecf2587ece0d8946fcd69b384cbee
by noreply＠mcs.anl.gov 27 Mar '13

27 Mar '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "C-Utils Repository". The branch, cleanup has been updated via 7e9cfef4ee5ecf2587ece0d8946fcd69b384cbee (commit) from 815cb2501574632a4e8122b6e58ecb9e21056e46 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 7e9cfef4ee5ecf2587ece0d8946fcd69b384cbee Author: Dries Kimpe <dkimpe(a)mcs.anl.gov> Date: Wed Mar 27 16:25:42 2013 -0500 Fix perftool logic ----------------------------------------------------------------------- Summary of changes: configure.ac | 7 ++++++- 1 files changed, 6 insertions(+), 1 deletions(-) Diff of changes: diff --git a/configure.ac b/configure.ac index fc19528..6568b6b 100644 --- a/configure.ac +++ b/configure.ac @@ -49,7 +49,12 @@ AC_ARG_WITH(perftools, [Location of google perftools installation]), PERFTOOL_DIR="$withval",PERFTOOL_DIR="") -if test -n "$PERFTOOL_DIR" ; +if test "X${PERFTOOL_DIR}" == "Xyes"; then + # if not specified, + PERFTOOL_DIR="/usr" +fi + +if test "X${PERFTOOL_DIR}" != "Xno"; then CPPFLAGS="$CPPFLAGS -I${PERFTOOL_DIR}/include" LDFLAGS="$LDFLAGS -L${PERFTOOL_DIR}/lib" hooks/post-receive -- C-Utils Repository

1 0

C-Utils Repository branch, cleanup, updated. 815cb2501574632a4e8122b6e58ecb9e21056e46
by noreply＠mcs.anl.gov 27 Mar '13

27 Mar '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "C-Utils Repository". The branch, cleanup has been updated via 815cb2501574632a4e8122b6e58ecb9e21056e46 (commit) via 5634360b9a82376e1e9cd8c1ff58f73e371f418f (commit) via 58786f549869b1fa86acfebeeaadb5a21e0118e1 (commit) from c0aef804222203b688f633ae8039440dfd07eeb6 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 815cb2501574632a4e8122b6e58ecb9e21056e46 Author: Dries Kimpe <dkimpe(a)mcs.anl.gov> Date: Wed Mar 27 16:08:23 2013 -0500 Disable TAP support in Automake commit 5634360b9a82376e1e9cd8c1ff58f73e371f418f Author: Dries Kimpe <dkimpe(a)mcs.anl.gov> Date: Wed Mar 27 16:02:35 2013 -0500 Make does VPATH redirect; no need srcdir commit 58786f549869b1fa86acfebeeaadb5a21e0118e1 Author: Dries Kimpe <dkimpe(a)mcs.anl.gov> Date: Wed Mar 27 16:02:09 2013 -0500 Automake installs tap-driver.sh for us ----------------------------------------------------------------------- Summary of changes: Makefile.am | 8 +- build-aux/tap-driver.sh | 651 ----------------------------------------------- configure.ac | 2 +- 3 files changed, 5 insertions(+), 656 deletions(-) delete mode 100755 build-aux/tap-driver.sh Diff of changes: diff --git a/Makefile.am b/Makefile.am index 979b32b..ffd7bcd 100644 --- a/Makefile.am +++ b/Makefile.am @@ -32,10 +32,10 @@ all-local: header-stamp check-local: header-stamp header-stamp: $(public_headers) - -$(MKDIR_P) $(top_builddir)/include + -$(MKDIR_P) $(builddir)/include -( \ for a in $^; do\ - cp $(top_srcdir)/$$a $(top_builddir)/include ; \ + cp $$a $(builddir)/include ; \ done ;\ ) -touch header-stamp @@ -47,8 +47,8 @@ clean-local: # note: requires automake 1.12 or above # Ignored on older versions -TEST_LOG_DRIVER = env AM_TAP_AWK='$(AWK)' $(SHELL) \ - $(top_srcdir)/build-aux/tap-driver.sh +#TEST_LOG_DRIVER = env AM_TAP_AWK='$(AWK)' $(SHELL) \ +# $(top_srcdir)/build-aux/tap-driver.sh #----------------------------------------------------- diff --git a/build-aux/tap-driver.sh b/build-aux/tap-driver.sh deleted file mode 100755 index 5014d41..0000000 --- a/build-aux/tap-driver.sh +++ /dev/null @@ -1,651 +0,0 @@ -#! /bin/sh -# Copyright (C) 2011-2013 Free Software Foundation, Inc. -# -# This program is free software; you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation; either version 2, or (at your option) -# any later version. -# -# This program is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with this program. If not, see <http://www.gnu.org/licenses/>. - -# As a special exception to the GNU General Public License, if you -# distribute this file as part of a program that contains a -# configuration script generated by Autoconf, you may include it under -# the same distribution terms that you use for the rest of that program. - -# This file is maintained in Automake, please report -# bugs to <bug-automake(a)gnu.org> or send patches to -# <automake-patches(a)gnu.org>. - -scriptversion=2011-12-27.17; # UTC - -# Make unconditional expansion of undefined variables an error. This -# helps a lot in preventing typo-related bugs. -set -u - -me=tap-driver.sh - -fatal () -{ - echo "$me: fatal: $*" >&2 - exit 1 -} - -usage_error () -{ - echo "$me: $*" >&2 - print_usage >&2 - exit 2 -} - -print_usage () -{ - cat <<END -Usage: - tap-driver.sh --test-name=NAME --log-file=PATH --trs-file=PATH - [--expect-failure={yes|no}] [--color-tests={yes|no}] - [--enable-hard-errors={yes|no}] [--ignore-exit] - [--diagnostic-string=STRING] [--merge|--no-merge] - [--comments|--no-comments] [--] TEST-COMMAND -The \`--test-name', \`--log-file' and \`--trs-file' options are mandatory. -END } - -# TODO: better error handling in option parsing (in particular, ensure -# TODO: $log_file, $trs_file and $test_name are defined). -test_name= # Used for reporting. -log_file= # Where to save the result and output of the test script. -trs_file= # Where to save the metadata of the test run. -expect_failure=0 -color_tests=0 -merge=0 -ignore_exit=0 -comments=0 -diag_string='#' -while test $# -gt 0; do - case $1 in - --help) print_usage; exit $?;; - --version) echo "$me $scriptversion"; exit $?;; - --test-name) test_name=$2; shift;; - --log-file) log_file=$2; shift;; - --trs-file) trs_file=$2; shift;; - --color-tests) color_tests=$2; shift;; - --expect-failure) expect_failure=$2; shift;; - --enable-hard-errors) shift;; # No-op. - --merge) merge=1;; - --no-merge) merge=0;; - --ignore-exit) ignore_exit=1;; - --comments) comments=1;; - --no-comments) comments=0;; - --diagnostic-string) diag_string=$2; shift;; - --) shift; break;; - -*) usage_error "invalid option: '$1'";; - esac - shift -done - -test $# -gt 0 || usage_error "missing test command" - -case $expect_failure in - yes) expect_failure=1;; - *) expect_failure=0;; -esac - -if test $color_tests = yes; then - init_colors=' - color_map["red"]="[0;31m" # Red. - color_map["grn"]="[0;32m" # Green. - color_map["lgn"]="[1;32m" # Light green. - color_map["blu"]="[1;34m" # Blue. - color_map["mgn"]="[0;35m" # Magenta. - color_map["std"]="[m" # No color. - color_for_result["ERROR"] = "mgn" - color_for_result["PASS"] = "grn" - color_for_result["XPASS"] = "red" - color_for_result["FAIL"] = "red" - color_for_result["XFAIL"] = "lgn" - color_for_result["SKIP"] = "blu"' -else - init_colors='' -fi - -# :; is there to work around a bug in bash 3.2 (and earlier) which -# does not always set '$?' properly on redirection failure. -# See the Autoconf manual for more details. -:;{ - ( - # Ignore common signals (in this subshell only!), to avoid potential - # problems with Korn shells. Some Korn shells are known to propagate - # to themselves signals that have killed a child process they were - # waiting for; this is done at least for SIGINT (and usually only for - # it, in truth). Without the `trap' below, such a behaviour could - # cause a premature exit in the current subshell, e.g., in case the - # test command it runs gets terminated by a SIGINT. Thus, the awk - # script we are piping into would never seen the exit status it - # expects on its last input line (which is displayed below by the - # last `echo $?' statement), and would thus die reporting an internal - # error. - # For more information, see the Autoconf manual and the threads: - # <http://lists.gnu.org/archive/html/bug-autoconf/2011-09/msg00004.html> - # <http://mail.opensolaris.org/pipermail/ksh93-integration-discuss/2009-Februa…> - trap : 1 3 2 13 15 - if test $merge -gt 0; then - exec 2>&1 - else - exec 2>&3 - fi - "$@" - echo $? - ) | LC_ALL=C ${AM_TAP_AWK-awk} \ - -v me="$me" \ - -v test_script_name="$test_name" \ - -v log_file="$log_file" \ - -v trs_file="$trs_file" \ - -v expect_failure="$expect_failure" \ - -v merge="$merge" \ - -v ignore_exit="$ignore_exit" \ - -v comments="$comments" \ - -v diag_string="$diag_string" \ -' -# FIXME: the usages of "cat >&3" below could be optimized when using -# FIXME: GNU awk, and/on on systems that supports /dev/fd/. - -# Implementation note: in what follows, `result_obj` will be an -# associative array that (partly) simulates a TAP result object -# from the `TAP::Parser` perl module. - -## ----------- ## -## FUNCTIONS ## -## ----------- ## - -function fatal(msg) -{ - print me ": " msg | "cat >&2" - exit 1 -} - -function abort(where) -{ - fatal("internal error " where) -} - -# Convert a boolean to a "yes"/"no" string. -function yn(bool) -{ - return bool ? "yes" : "no"; -} - -function add_test_result(result) -{ - if (!test_results_index) - test_results_index = 0 - test_results_list[test_results_index] = result - test_results_index += 1 - test_results_seen[result] = 1; -} - -# Whether the test script should be re-run by "make recheck". -function must_recheck() -{ - for (k in test_results_seen) - if (k != "XFAIL" && k != "PASS" && k != "SKIP") - return 1 - return 0 -} - -# Whether the content of the log file associated to this test should -# be copied into the "global" test-suite.log. -function copy_in_global_log() -{ - for (k in test_results_seen) - if (k != "PASS") - return 1 - return 0 -} - -# FIXME: this can certainly be improved ... -function get_global_test_result() -{ - if ("ERROR" in test_results_seen) - return "ERROR" - if ("FAIL" in test_results_seen || "XPASS" in test_results_seen) - return "FAIL" - all_skipped = 1 - for (k in test_results_seen) - if (k != "SKIP") - all_skipped = 0 - if (all_skipped) - return "SKIP" - return "PASS"; -} - -function stringify_result_obj(result_obj) -{ - if (result_obj["is_unplanned"] || result_obj["number"] != testno) - return "ERROR" - - if (plan_seen == LATE_PLAN) - return "ERROR" - - if (result_obj["directive"] == "TODO") - return result_obj["is_ok"] ? "XPASS" : "XFAIL" - - if (result_obj["directive"] == "SKIP") - return result_obj["is_ok"] ? "SKIP" : COOKED_FAIL; - - if (length(result_obj["directive"])) - abort("in function stringify_result_obj()") - - return result_obj["is_ok"] ? COOKED_PASS : COOKED_FAIL -} - -function decorate_result(result) -{ - color_name = color_for_result[result] - if (color_name) - return color_map[color_name] "" result "" color_map["std"] - # If we are not using colorized output, or if we do not know how - # to colorize the given result, we should return it unchanged. - return result -} - -function report(result, details) -{ - if (result ~ /^(X?(PASS|FAIL)|SKIP|ERROR)/) - { - msg = ": " test_script_name - add_test_result(result) - } - else if (result == "#") - { - msg = " " test_script_name ":" - } - else - { - abort("in function report()") - } - if (length(details)) - msg = msg " " details - # Output on console might be colorized. - print decorate_result(result) msg - # Log the result in the log file too, to help debugging (this is - # especially true when said result is a TAP error or "Bail out!"). - print result msg | "cat >&3"; -} - -function testsuite_error(error_message) -{ - report("ERROR", "- " error_message) -} - -function handle_tap_result() -{ - details = result_obj["number"]; - if (length(result_obj["description"])) - details = details " " result_obj["description"] - - if (plan_seen == LATE_PLAN) - { - details = details " # AFTER LATE PLAN"; - } - else if (result_obj["is_unplanned"]) - { - details = details " # UNPLANNED"; - } - else if (result_obj["number"] != testno) - { - details = sprintf("%s # OUT-OF-ORDER (expecting %d)", - details, testno); - } - else if (result_obj["directive"]) - { - details = details " # " result_obj["directive"]; - if (length(result_obj["explanation"])) - details = details " " result_obj["explanation"] - } - - report(stringify_result_obj(result_obj), details) -} - -# `skip_reason` should be empty whenever planned > 0. -function handle_tap_plan(planned, skip_reason) -{ - planned += 0 # Avoid getting confused if, say, `planned` is "00" - if (length(skip_reason) && planned > 0) - abort("in function handle_tap_plan()") - if (plan_seen) - { - # Error, only one plan per stream is acceptable. - testsuite_error("multiple test plans") - return; - } - planned_tests = planned - # The TAP plan can come before or after *all* the TAP results; we speak - # respectively of an "early" or a "late" plan. If we see the plan line - # after at least one TAP result has been seen, assume we have a late - # plan; in this case, any further test result seen after the plan will - # be flagged as an error. - plan_seen = (testno >= 1 ? LATE_PLAN : EARLY_PLAN) - # If testno > 0, we have an error ("too many tests run") that will be - # automatically dealt with later, so do not worry about it here. If - # $plan_seen is true, we have an error due to a repeated plan, and that - # has already been dealt with above. Otherwise, we have a valid "plan - # with SKIP" specification, and should report it as a particular kind - # of SKIP result. - if (planned == 0 && testno == 0) - { - if (length(skip_reason)) - skip_reason = "- " skip_reason; - report("SKIP", skip_reason); - } -} - -function extract_tap_comment(line) -{ - if (index(line, diag_string) == 1) - { - # Strip leading `diag_string` from `line`. - line = substr(line, length(diag_string) + 1) - # And strip any leading and trailing whitespace left. - sub("^[ \t]*", "", line) - sub("[ \t]*$", "", line) - # Return what is left (if any). - return line; - } - return ""; -} - -# When this function is called, we know that line is a TAP result line, -# so that it matches the (perl) RE "^(not )?ok\b". -function setup_result_obj(line) -{ - # Get the result, and remove it from the line. - result_obj["is_ok"] = (substr(line, 1, 2) == "ok" ? 1 : 0) - sub("^(not )?ok[ \t]*", "", line) - - # If the result has an explicit number, get it and strip it; otherwise, - # automatically assing the next progresive number to it. - if (line ~ /^[0-9]+$/ || line ~ /^[0-9]+[^a-zA-Z0-9_]/) - { - match(line, "^[0-9]+") - # The final `+ 0` is to normalize numbers with leading zeros. - result_obj["number"] = substr(line, 1, RLENGTH) + 0 - line = substr(line, RLENGTH + 1) - } - else - { - result_obj["number"] = testno - } - - if (plan_seen == LATE_PLAN) - # No further test results are acceptable after a "late" TAP plan - # has been seen. - result_obj["is_unplanned"] = 1 - else if (plan_seen && testno > planned_tests) - result_obj["is_unplanned"] = 1 - else - result_obj["is_unplanned"] = 0 - - # Strip trailing and leading whitespace. - sub("^[ \t]*", "", line) - sub("[ \t]*$", "", line) - - # This will have to be corrected if we have a "TODO"/"SKIP" directive. - result_obj["description"] = line - result_obj["directive"] = "" - result_obj["explanation"] = "" - - if (index(line, "#") == 0) - return # No possible directive, nothing more to do. - - # Directives are case-insensitive. - rx = "[ \t]*#[ \t]*([tT][oO][dD][oO]|[sS][kK][iI][pP])[ \t]*" - - # See whether we have the directive, and if yes, where. - pos = match(line, rx "$") - if (!pos) - pos = match(line, rx "[^a-zA-Z0-9_]") - - # If there was no TAP directive, we have nothing more to do. - if (!pos) - return - - # Let`s now see if the TAP directive has been escaped. For example: - # escaped: ok \# SKIP - # not escaped: ok \\# SKIP - # escaped: ok \\\\\# SKIP - # not escaped: ok \ # SKIP - if (substr(line, pos, 1) == "#") - { - bslash_count = 0 - for (i = pos; i > 1 && substr(line, i - 1, 1) == "\\"; i--) - bslash_count += 1 - if (bslash_count % 2) - return # Directive was escaped. - } - - # Strip the directive and its explanation (if any) from the test - # description. - result_obj["description"] = substr(line, 1, pos - 1) - # Now remove the test description from the line, that has been dealt - # with already. - line = substr(line, pos) - # Strip the directive, and save its value (normalized to upper case). - sub("^[ \t]*#[ \t]*", "", line) - result_obj["directive"] = toupper(substr(line, 1, 4)) - line = substr(line, 5) - # Now get the explanation for the directive (if any), with leading - # and trailing whitespace removed. - sub("^[ \t]*", "", line) - sub("[ \t]*$", "", line) - result_obj["explanation"] = line -} - -function get_test_exit_message(status) -{ - if (status == 0) - return "" - if (status !~ /^[1-9][0-9]*$/) - abort("getting exit status") - if (status < 127) - exit_details = "" - else if (status == 127) - exit_details = " (command not found?)" - else if (status >= 128 && status <= 255) - exit_details = sprintf(" (terminated by signal %d?)", status - 128) - else if (status > 256 && status <= 384) - # We used to report an "abnormal termination" here, but some Korn - # shells, when a child process die due to signal number n, can leave - # in $? an exit status of 256+n instead of the more standard 128+n. - # Apparently, both behaviours are allowed by POSIX (2008), so be - # prepared to handle them both. See also Austing Group report ID - # 0000051 <http://www.austingroupbugs.net/view.php?id=51> - exit_details = sprintf(" (terminated by signal %d?)", status - 256) - else - # Never seen in practice. - exit_details = " (abnormal termination)" - return sprintf("exited with status %d%s", status, exit_details) -} - -function write_test_results() -{ - print ":global-test-result: " get_global_test_result() > trs_file - print ":recheck: " yn(must_recheck()) > trs_file - print ":copy-in-global-log: " yn(copy_in_global_log()) > trs_file - for (i = 0; i < test_results_index; i += 1) - print ":test-result: " test_results_list[i] > trs_file - close(trs_file); -} - -BEGIN { - -## ------- ## -## SETUP ## -## ------- ## - -'"$init_colors"' - -# Properly initialized once the TAP plan is seen. -planned_tests = 0 - -COOKED_PASS = expect_failure ? "XPASS": "PASS"; -COOKED_FAIL = expect_failure ? "XFAIL": "FAIL"; - -# Enumeration-like constants to remember which kind of plan (if any) -# has been seen. It is important that NO_PLAN evaluates "false" as -# a boolean. -NO_PLAN = 0 -EARLY_PLAN = 1 -LATE_PLAN = 2 - -testno = 0 # Number of test results seen so far. -bailed_out = 0 # Whether a "Bail out!" directive has been seen. - -# Whether the TAP plan has been seen or not, and if yes, which kind -# it is ("early" is seen before any test result, "late" otherwise). -plan_seen = NO_PLAN - -## --------- ## -## PARSING ## -## --------- ## - -is_first_read = 1 - -while (1) - { - # Involutions required so that we are able to read the exit status - # from the last input line. - st = getline - if (st < 0) # I/O error. - fatal("I/O error while reading from input stream") - else if (st == 0) # End-of-input - { - if (is_first_read) - abort("in input loop: only one input line") - break - } - if (is_first_read) - { - is_first_read = 0 - nextline = $0 - continue - } - else - { - curline = nextline - nextline = $0 - $0 = curline - } - # Copy any input line verbatim into the log file. - print | "cat >&3" - # Parsing of TAP input should stop after a "Bail out!" directive. - if (bailed_out) - continue - - # TAP test result. - if ($0 ~ /^(not )?ok$/ || $0 ~ /^(not )?ok[^a-zA-Z0-9_]/) - { - testno += 1 - setup_result_obj($0) - handle_tap_result() - } - # TAP plan (normal or "SKIP" without explanation). - else if ($0 ~ /^1\.\.[0-9]+[ \t]*$/) - { - # The next two lines will put the number of planned tests in $0. - sub("^1\\.\\.", "") - sub("[^0-9]*$", "") - handle_tap_plan($0, "") - continue - } - # TAP "SKIP" plan, with an explanation. - else if ($0 ~ /^1\.\.0+[ \t]*#/) - { - # The next lines will put the skip explanation in $0, stripping - # any leading and trailing whitespace. This is a little more - # tricky in truth, since we want to also strip a potential leading - # "SKIP" string from the message. - sub("^[^#]*#[ \t]*(SKIP[: \t][ \t]*)?", "") - sub("[ \t]*$", ""); - handle_tap_plan(0, $0) - } - # "Bail out!" magic. - # Older versions of prove and TAP::Harness (e.g., 3.17) did not - # recognize a "Bail out!" directive when preceded by leading - # whitespace, but more modern versions (e.g., 3.23) do. So we - # emulate the latter, "more modern" behaviour. - else if ($0 ~ /^[ \t]*Bail out!/) - { - bailed_out = 1 - # Get the bailout message (if any), with leading and trailing - # whitespace stripped. The message remains stored in `$0`. - sub("^[ \t]*Bail out![ \t]*", ""); - sub("[ \t]*$", ""); - # Format the error message for the - bailout_message = "Bail out!" - if (length($0)) - bailout_message = bailout_message " " $0 - testsuite_error(bailout_message) - } - # Maybe we have too look for dianogtic comments too. - else if (comments != 0) - { - comment = extract_tap_comment($0); - if (length(comment)) - report("#", comment); - } - } - -## -------- ## -## FINISH ## -## -------- ## - -# A "Bail out!" directive should cause us to ignore any following TAP -# error, as well as a non-zero exit status from the TAP producer. -if (!bailed_out) - { - if (!plan_seen) - { - testsuite_error("missing test plan") - } - else if (planned_tests != testno) - { - bad_amount = testno > planned_tests ? "many" : "few" - testsuite_error(sprintf("too %s tests run (expected %d, got %d)", - bad_amount, planned_tests, testno)) - } - if (!ignore_exit) - { - # Fetch exit status from the last line. - exit_message = get_test_exit_message(nextline) - if (exit_message) - testsuite_error(exit_message) - } - } - -write_test_results() - -exit 0 - -} # End of "BEGIN" block. -' - -# TODO: document that we consume the file descriptor 3 :-( -} 3>"$log_file" - -test $? -eq 0 || fatal "I/O or internal error" - -# Local Variables: -# mode: shell-script -# sh-indentation: 2 -# eval: (add-hook 'write-file-hooks 'time-stamp) -# time-stamp-start: "scriptversion=" -# time-stamp-format: "%:y-%02m-%02d.%02H" -# time-stamp-time-zone: "UTC" -# time-stamp-end: "; # UTC" -# End: diff --git a/configure.ac b/configure.ac index 60384f1..fc19528 100644 --- a/configure.ac +++ b/configure.ac @@ -20,7 +20,7 @@ AC_PROG_AWK AC_PROG_LN_S AS_MKDIR_P -AC_REQUIRE_AUX_FILE([tap-driver.sh]) +dnl AC_REQUIRE_AUX_FILE([tap-driver.sh]) AC_CHECK_SIZEOF([long int]) hooks/post-receive -- C-Utils Repository

1 0

C-Utils Repository branch, cleanup, updated. c0aef804222203b688f633ae8039440dfd07eeb6
by noreply＠mcs.anl.gov 27 Mar '13

27 Mar '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "C-Utils Repository". The branch, cleanup has been updated discards 65df0352a6fbadb8beb5c353ac8d4a0390767d1d (commit) via c0aef804222203b688f633ae8039440dfd07eeb6 (commit) This update added new revisions after undoing existing revisions. That is to say, the old revision is not a strict subset of the new revision. This situation occurs when you --force push a change and generate a repository containing something like this: * -- * -- B -- O -- O -- O (65df0352a6fbadb8beb5c353ac8d4a0390767d1d) \ N -- N -- N (c0aef804222203b688f633ae8039440dfd07eeb6) When this happens we assume that you've already had alert emails for all of the O revisions, and so we here report only the revisions in the N branch from the common base, B. Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit c0aef804222203b688f633ae8039440dfd07eeb6 Author: Dries Kimpe <dkimpe(a)mcs.anl.gov> Date: Wed Mar 27 14:20:58 2013 -0500 Switch c-utils over to Automake - VPATH build working - Puts all public header files in $(builddir)/include and library into $(builddir)/lib to simplify compiling against an uninstalled version of c-utils - make dist & make distcheck working - made libev into convenience library so it can be compiled with additional flags (such as ignore warnings) ----------------------------------------------------------------------- Summary of changes: Makefile.am | 28 +++++++++++++++++++++++++++- build-aux/tap-driver.sh | 5 ++--- configure.ac | 4 +++- libev/Makefile.subdir | 4 ++-- src/Makefile.subdir | 18 +++++++++--------- 5 files changed, 43 insertions(+), 16 deletions(-) Diff of changes: diff --git a/Makefile.am b/Makefile.am index 0dc12af..979b32b 100644 --- a/Makefile.am +++ b/Makefile.am @@ -15,8 +15,34 @@ EXTRA_DIST = COPYRIGHT TODO nobase_include_HEADERS= +#------------------------------------------------------------ + +# List public headers in autoconf @public_headers@ variable +# In addition to being installed, +# this makes sure that copies of (or links to) these headers +# will exist in the build include directory to make embedding c-utils +# easier. #----------------------------------------------------- -AM_CPPFLAGS += -I$(srcdir)/include + +public_headers= + +include_HEADERS += $(public_headers) + +all-local: header-stamp +check-local: header-stamp + +header-stamp: $(public_headers) + -$(MKDIR_P) $(top_builddir)/include + -( \ + for a in $^; do\ + cp $(top_srcdir)/$$a $(top_builddir)/include ; \ + done ;\ + ) + -touch header-stamp + +clean-local: + -rm header-stamp + #----------------------------------------------------- # note: requires automake 1.12 or above diff --git a/build-aux/tap-driver.sh b/build-aux/tap-driver.sh index 259ca0d..5014d41 100755 --- a/build-aux/tap-driver.sh +++ b/build-aux/tap-driver.sh @@ -1,5 +1,5 @@ #! /bin/sh -# Copyright (C) 2011-2012 Free Software Foundation, Inc. +# Copyright (C) 2011-2013 Free Software Foundation, Inc. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -54,8 +54,7 @@ Usage: [--diagnostic-string=STRING] [--merge|--no-merge] [--comments|--no-comments] [--] TEST-COMMAND The \`--test-name', \`--log-file' and \`--trs-file' options are mandatory. -END -} +END } # TODO: better error handling in option parsing (in particular, ensure # TODO: $log_file, $trs_file and $test_name are defined). diff --git a/configure.ac b/configure.ac index 52490f0..60384f1 100644 --- a/configure.ac +++ b/configure.ac @@ -9,7 +9,7 @@ AC_CONFIG_AUX_DIR([build-aux]) AM_INIT_AUTOMAKE([foreign subdir-objects -Wall]) m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) -AC_CONFIG_SRCDIR([include/c-utils-common.h]) +AC_CONFIG_SRCDIR([src/c-utils-common.h]) AC_CONFIG_HEADER([c-utils-config.h]) AC_PROG_CC @@ -17,6 +17,8 @@ AM_PROG_CC_C_O AM_PROG_AR AC_PROG_LIBTOOL AC_PROG_AWK +AC_PROG_LN_S +AS_MKDIR_P AC_REQUIRE_AUX_FILE([tap-driver.sh]) diff --git a/libev/Makefile.subdir b/libev/Makefile.subdir index e4ffc09..7a2b9f0 100644 --- a/libev/Makefile.subdir +++ b/libev/Makefile.subdir @@ -26,6 +26,6 @@ noinst_HEADERS += \ libev/ev_select.c\ libev/ev_win32.c -include_HEADERS += \ - include/ev.h +public_headers += \ + libev/ev.h diff --git a/src/Makefile.subdir b/src/Makefile.subdir index 99d09d0..dc29495 100644 --- a/src/Makefile.subdir +++ b/src/Makefile.subdir @@ -4,13 +4,13 @@ src/lookup3.c\ src/triton-string.c\ src/workqueue.c -include_HEADERS += include/c-utils-common.h \ - include/triton-base.h \ - include/triton-list.h \ - include/triton-string.h \ - include/triton-thread.h \ - include/triton-hash.h \ - include/triton-uint128.h \ - include/workqueue.h \ - include/jenkins-hash.h +public_headers += src/c-utils-common.h \ + src/triton-base.h \ + src/triton-list.h \ + src/triton-string.h \ + src/triton-thread.h \ + src/triton-hash.h \ + src/triton-uint128.h \ + src/workqueue.h \ + src/jenkins-hash.h hooks/post-receive -- C-Utils Repository

1 0

C-Utils Repository branch, cleanup, created. 65df0352a6fbadb8beb5c353ac8d4a0390767d1d
by noreply＠mcs.anl.gov 27 Mar '13

27 Mar '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "C-Utils Repository". The branch, cleanup has been created at 65df0352a6fbadb8beb5c353ac8d4a0390767d1d (commit) - Log ----------------------------------------------------------------- commit 65df0352a6fbadb8beb5c353ac8d4a0390767d1d Author: Dries Kimpe <dkimpe(a)mcs.anl.gov> Date: Wed Mar 27 14:20:58 2013 -0500 Switch c-utils over to Automake - VPATH build working - Puts all public header files in $(builddir)/include and library into $(builddir)/lib to simplify compiling against an uninstalled version of c-utils - make dist & make distcheck working - made libev into convenience library so it can be compiled with additional flags (such as ignore warnings) ----------------------------------------------------------------------- hooks/post-receive -- C-Utils Repository

1 0

branch, ticket-227, updated. cf5f47a33038eecaea3f59140354161b20dd5da0
by noreply＠mcs.anl.gov 06 Mar '13

06 Mar '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, ticket-227 has been updated via cf5f47a33038eecaea3f59140354161b20dd5da0 (commit) from 28cd8ccafe302be7ac8fa6465aa87e95edf97e33 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit cf5f47a33038eecaea3f59140354161b20dd5da0 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Mar 6 22:26:26 2013 -0500 fix mutex bug in mpi-method ----------------------------------------------------------------------- Summary of changes: code/src/net/mpi/mpi-method.ae | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) Diff of changes: diff --git a/code/src/net/mpi/mpi-method.ae b/code/src/net/mpi/mpi-method.ae index 7ebf260..bfd3abb 100644 --- a/code/src/net/mpi/mpi-method.ae +++ b/code/src/net/mpi/mpi-method.ae @@ -1897,6 +1897,7 @@ static void* thread_fn(void* arg) aesop_sem_up(&nb_req->sem); } } + triton_mutex_lock(&thread_mutex); } triton_mutex_unlock(&thread_mutex); hooks/post-receive --

1 0

branch, ticket-227, updated. 28cd8ccafe302be7ac8fa6465aa87e95edf97e33
by noreply＠mcs.anl.gov 06 Mar '13

06 Mar '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "". The branch, ticket-227 has been updated via 28cd8ccafe302be7ac8fa6465aa87e95edf97e33 (commit) from 06c6c97e17dc63fc94627c0cfb82a8e09e092efc (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 28cd8ccafe302be7ac8fa6465aa87e95edf97e33 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Mar 6 22:08:30 2013 -0500 destroy strings before re-initializing ----------------------------------------------------------------------- Summary of changes: code/src/common/triton-bootstrap.ae | 8 ++++++-- 1 files changed, 6 insertions(+), 2 deletions(-) Diff of changes: diff --git a/code/src/common/triton-bootstrap.ae b/code/src/common/triton-bootstrap.ae index 950ccf9..271a556 100644 --- a/code/src/common/triton-bootstrap.ae +++ b/code/src/common/triton-bootstrap.ae @@ -147,6 +147,9 @@ __blocking triton_ret_t triton_bootstrap_by_hostfile(const char *filename) return(tret); } + triton_string_destroy(&key); + triton_string_destroy(&value); + triton_string_init(&key, "triton.state"); triton_string_init(&value, "TRITON_STATUS_PROTO"); tret = triton_state_update(node, &key, &value); @@ -156,10 +159,11 @@ __blocking triton_ret_t triton_bootstrap_by_hostfile(const char *filename) triton_string_destroy(&value); return(tret); } + + triton_string_destroy(&key); + triton_string_destroy(&value); } - triton_string_destroy(&key); - triton_string_destroy(&value); fclose(file); return TRITON_SUCCESS; hooks/post-receive --

1 0

C-Utils Repository branch, master, updated. dd55918cf621cb0767dd2351830e150310a4d20e
by noreply＠mcs.anl.gov 06 Mar '13

06 Mar '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "C-Utils Repository". The branch, master has been updated via dd55918cf621cb0767dd2351830e150310a4d20e (commit) via 87ee58e1b38b0cf415d6a8dc73ff02c012df14ff (commit) from 804b3af17ac4b868530b5b3e95fbf978b7306729 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit dd55918cf621cb0767dd2351830e150310a4d20e Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Mar 6 22:08:16 2013 -0500 work around some mutex problems commit 87ee58e1b38b0cf415d6a8dc73ff02c012df14ff Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Mar 6 21:33:06 2013 -0500 fix locking bug in triton-string.h don't destroy and re-init mutex while holding mutex ----------------------------------------------------------------------- Summary of changes: triton-string.h | 18 ++++++++++++++---- 1 files changed, 14 insertions(+), 4 deletions(-) Diff of changes: diff --git a/triton-string.h b/triton-string.h index b90bda5..6c1926b 100644 --- a/triton-string.h +++ b/triton-string.h @@ -136,15 +136,19 @@ static inline const char* triton_string_get(triton_string_t* str) #define triton_string_get(__str__) ((__str__)->const_string ? (__str__)->const_string : (__str__)->string) #endif -#define triton_string_set_null(__val) do { \ +#define triton_string_clear(__val) do { \ (__val)->parent = NULL; \ (__val)->refcount = 0; \ - triton_mutex_init(&(__val)->reflock, NULL); \ (__val)->size = 0; \ (__val)->string = NULL; \ (__val)->const_string = NULL; \ } while(0) +#define triton_string_set_null(__val) do { \ + triton_string_clear(__val); \ + triton_mutex_init(&(__val)->reflock, NULL); \ +} while(0) + static inline void triton_string_decrement(triton_string_t *s) { triton_mutex_lock(&s->reflock); @@ -165,10 +169,16 @@ static inline void triton_string_destroy(triton_string_t *s) { if(s->string) free(s->string); + triton_mutex_unlock(&s->reflock); triton_mutex_destroy(&s->reflock); - triton_string_set_null(s); + triton_string_clear(s); } - triton_mutex_unlock(&s->reflock); + else + { + triton_mutex_unlock(&s->reflock); + } + + return; } static inline void triton_string_copy(triton_string_t *to, triton_string_t *from) hooks/post-receive -- C-Utils Repository

1 0

C-Utils Repository branch, master, updated. 804b3af17ac4b868530b5b3e95fbf978b7306729
by noreply＠mcs.anl.gov 06 Mar '13

06 Mar '13

This is an automated email from the git hooks/post-receive script. It was generated because a ref change was pushed to the repository containing the project "C-Utils Repository". The branch, master has been updated via 804b3af17ac4b868530b5b3e95fbf978b7306729 (commit) from e93a16e8704bff24974622ce3aa79d88944ad786 (commit) Those revisions listed above that are new to this repository have not appeared on any other notification email; so we list those revisions in full, below. - Log ----------------------------------------------------------------- commit 804b3af17ac4b868530b5b3e95fbf978b7306729 Author: Phil Carns <carns(a)mcs.anl.gov> Date: Wed Mar 6 20:38:34 2013 -0500 revert 613eabc3a1585a1920843f711a848fbffe3ebf2c Makefile.in too ----------------------------------------------------------------------- Summary of changes: Makefile.in | 11 ----------- 1 files changed, 0 insertions(+), 11 deletions(-) Diff of changes: diff --git a/Makefile.in b/Makefile.in index 2ccded5..ae9fe99 100644 --- a/Makefile.in +++ b/Makefile.in @@ -310,16 +310,5 @@ etags: | xargs etags install:: all - install -d $(bindir) install -d $(libdir) - install -d $(includedir) install -m 644 libc-utils.* $(libdir) - install -m 644 $(srcdir)/triton-thread.h $(includedir) - install -m 644 $(srcdir)/triton-list.h $(includedir) - install -m 644 $(srcdir)/jenkins-hash.h $(includedir) - install -m 644 $(srcdir)/triton-hash.h $(includedir) - install -m 644 $(srcdir)/triton-uint128.h $(includedir) - install -m 644 $(srcdir)/triton-string.h $(includedir) - install -m 644 $(srcdir)/triton-base.h $(includedir) -# install -m 644 c-utils-config.h $(includedir) - install -m 644 c-utils-settings.h $(includedir) hooks/post-receive -- C-Utils Repository

1 0