draft-ietf-decade-arch-00.txt   draft-ietf-decade-arch-01.txt 
DECADE R. Alimi DECADE R. Alimi
Internet-Draft Google Internet-Draft Google
Intended status: Standards Track Y. Yang Intended status: Informational Y. Yang
Expires: September 8, 2011 Yale University Expires: November 22, 2011 Yale University
A. Rahman A. Rahman
InterDigital Communications, LLC InterDigital Communications, LLC
D. Kutscher D. Kutscher
NEC NEC
H. Liu H. Liu
Yale University Yale University
March 7, 2011 May 21, 2011
DECADE Architecture DECADE Architecture
draft-ietf-decade-arch-00 draft-ietf-decade-arch-01
Abstract Abstract
Peer-to-peer (P2P) applications have become widely used on the Peer-to-peer (P2P) applications have become widely used on the
Internet today and make up a large portion of the traffic in many Internet today and make up a large portion of the traffic in many
networks. One technique to improve the network efficiency of P2P networks. One technique to improve the network efficiency of P2P
applications is to introduce storage capabilities within the applications is to introduce storage capabilities within the
networks. The DECADE Working Group has been formed with the goal of networks. The DECADE Working Group has been formed with the goal of
developing an architecture to provide this capability. This document developing an architecture to provide this capability. This document
presents an architecture, discusses the underlying principles, and presents an architecture, discusses the underlying principles, and
skipping to change at page 2, line 5 skipping to change at page 2, line 5
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt. http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
This Internet-Draft will expire on September 8, 2011. This Internet-Draft will expire on November 22, 2011.
Copyright Notice Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 3, line 16 skipping to change at page 3, line 16
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5
2. Entities . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2. Entities . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.1. DECADE Storage Servers . . . . . . . . . . . . . . . . . . 6 2.1. DECADE Storage Servers . . . . . . . . . . . . . . . . . . 6
2.2. DECADE Storage Provider . . . . . . . . . . . . . . . . . 6 2.2. DECADE Storage Provider . . . . . . . . . . . . . . . . . 6
2.3. DECADE Content Providers . . . . . . . . . . . . . . . . . 6 2.3. DECADE Content Providers . . . . . . . . . . . . . . . . . 6
2.4. DECADE Content Consumers . . . . . . . . . . . . . . . . . 6 2.4. DECADE Content Consumers . . . . . . . . . . . . . . . . . 6
2.5. Content Distribution Application . . . . . . . . . . . . . 6 2.5. Content Distribution Application . . . . . . . . . . . . . 6
2.6. Application End-Point . . . . . . . . . . . . . . . . . . 7 2.6. Application End-Point . . . . . . . . . . . . . . . . . . 7
3. Architectural Principles . . . . . . . . . . . . . . . . . . . 7 3. Architectural Principles . . . . . . . . . . . . . . . . . . . 7
3.1. Decoupled Control and Data Planes . . . . . . . . . . . . 7 3.1. Decoupled Control/Metadata and Data Planes . . . . . . . . 7
3.2. Immutable Data Objects . . . . . . . . . . . . . . . . . . 8 3.2. Immutable Data Objects . . . . . . . . . . . . . . . . . . 8
3.3. Data Object Identifiers . . . . . . . . . . . . . . . . . 9 3.3. Data Object Identifiers . . . . . . . . . . . . . . . . . 9
3.4. Explicit Control . . . . . . . . . . . . . . . . . . . . . 9 3.4. Explicit Control . . . . . . . . . . . . . . . . . . . . . 10
3.5. Resource and Data Access Control through User 3.5. Resource and Data Access Control through User
Delegation . . . . . . . . . . . . . . . . . . . . . . . . 10 Delegation . . . . . . . . . . . . . . . . . . . . . . . . 10
3.5.1. Resource Allocation . . . . . . . . . . . . . . . . . 10 3.5.1. Resource Allocation . . . . . . . . . . . . . . . . . 10
3.5.2. User Delegations . . . . . . . . . . . . . . . . . . . 10 3.5.2. User Delegations . . . . . . . . . . . . . . . . . . . 10
4. System Components . . . . . . . . . . . . . . . . . . . . . . 11 4. System Components . . . . . . . . . . . . . . . . . . . . . . 11
4.1. Content Distribution Application . . . . . . . . . . . . . 13 4.1. Content Distribution Application . . . . . . . . . . . . . 13
4.1.1. Data Sequencing and Naming . . . . . . . . . . . . . . 13 4.1.1. Data Assembly . . . . . . . . . . . . . . . . . . . . 13
4.1.2. Native Protocols . . . . . . . . . . . . . . . . . . . 13 4.1.2. Native Protocols . . . . . . . . . . . . . . . . . . . 14
4.1.3. DECADE Client . . . . . . . . . . . . . . . . . . . . 14 4.1.3. DECADE Client . . . . . . . . . . . . . . . . . . . . 14
4.2. DECADE Server . . . . . . . . . . . . . . . . . . . . . . 14 4.2. DECADE Server . . . . . . . . . . . . . . . . . . . . . . 14
4.2.1. Access Control . . . . . . . . . . . . . . . . . . . . 14 4.2.1. Access Control . . . . . . . . . . . . . . . . . . . . 14
4.2.2. Resource Scheduling . . . . . . . . . . . . . . . . . 15 4.2.2. Resource Scheduling . . . . . . . . . . . . . . . . . 15
4.2.3. Data Store . . . . . . . . . . . . . . . . . . . . . . 15 4.2.3. Data Store . . . . . . . . . . . . . . . . . . . . . . 15
4.3. Protocols . . . . . . . . . . . . . . . . . . . . . . . . 15 4.3. Protocols . . . . . . . . . . . . . . . . . . . . . . . . 15
4.3.1. DECADE Resource Protocol . . . . . . . . . . . . . . . 15 4.3.1. DECADE Resource Protocol . . . . . . . . . . . . . . . 16
4.3.2. Standard Data Transports . . . . . . . . . . . . . . . 16 4.3.2. Standard Data Transports . . . . . . . . . . . . . . . 16
4.4. DECADE Data Sequencing and Naming . . . . . . . . . . . . 16 4.4. Data Sequencing and Naming . . . . . . . . . . . . . . . . 16
4.5. In-Network Storage Components Mapped to DECADE 4.4.1. DECADE Data Object Naming Schame . . . . . . . . . . . 16
Architecture . . . . . . . . . . . . . . . . . . . . . . . 17 4.4.2. Application Usage . . . . . . . . . . . . . . . . . . 17
4.5.1. Data Access Interface . . . . . . . . . . . . . . . . 17 4.4.3. Application Usage Example . . . . . . . . . . . . . . 17
4.5.2. Data Management Operations . . . . . . . . . . . . . . 17 4.5. Token-based Authentication and Resource Control . . . . . 19
4.5.3. Data Search Capability . . . . . . . . . . . . . . . . 17 4.6. In-Network Storage Components Mapped to DECADE
4.5.4. Access Control Authorization . . . . . . . . . . . . . 17 Architecture . . . . . . . . . . . . . . . . . . . . . . . 20
4.5.5. Resource Control Interface . . . . . . . . . . . . . . 17 4.6.1. Data Access Interface . . . . . . . . . . . . . . . . 20
4.5.6. Discovery Mechanism . . . . . . . . . . . . . . . . . 18 4.6.2. Data Management Operations . . . . . . . . . . . . . . 20
4.5.7. Storage Mode . . . . . . . . . . . . . . . . . . . . . 18 4.6.3. Data Search Capability . . . . . . . . . . . . . . . . 21
5. DECADE Protocols . . . . . . . . . . . . . . . . . . . . . . . 18 4.6.4. Access Control Authorization . . . . . . . . . . . . . 21
5.1. DECADE Resource Protocol (DRP) . . . . . . . . . . . . . . 18 4.6.5. Resource Control Interface . . . . . . . . . . . . . . 21
5.2. Standard Data Transport (SDT) . . . . . . . . . . . . . . 19 4.6.6. Discovery Mechanism . . . . . . . . . . . . . . . . . 21
5.2.1. Writing/Uploading Objects . . . . . . . . . . . . . . 19 4.6.7. Storage Mode . . . . . . . . . . . . . . . . . . . . . 21
5.2.2. Downloading Objects . . . . . . . . . . . . . . . . . 20 5. DECADE Protocols . . . . . . . . . . . . . . . . . . . . . . . 21
6. Server-to-Server Protocols . . . . . . . . . . . . . . . . . . 21 5.1. DECADE Resource Protocol (DRP) . . . . . . . . . . . . . . 22
6.1. Operational Overview . . . . . . . . . . . . . . . . . . . 21 5.1.1. Controlled Resources . . . . . . . . . . . . . . . . . 22
6.2. Potential Optimizations . . . . . . . . . . . . . . . . . 22 5.1.2. Token-based Authentication and Resource Control . . . 22
6.2.1. Pipelining to Avoid Store-and-Forward Delays . . . . . 22 5.1.3. Status Information . . . . . . . . . . . . . . . . . . 23
7. Security Considerations . . . . . . . . . . . . . . . . . . . 23 5.1.4. Object Properties . . . . . . . . . . . . . . . . . . 24
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 23 5.2. Standard Data Transport (SDT) . . . . . . . . . . . . . . 25
9. Informative References . . . . . . . . . . . . . . . . . . . . 23 5.2.1. Writing/Uploading Objects . . . . . . . . . . . . . . 25
Appendix A. Appendix: Evaluation of Candidate Existing 5.2.2. Downloading Objects . . . . . . . . . . . . . . . . . 26
Protocols for DECADE DRP and SDT . . . . . . . . . . 23 6. Server-to-Server Protocols . . . . . . . . . . . . . . . . . . 27
A.1. HTTP . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 6.1. Operational Overview . . . . . . . . . . . . . . . . . . . 27
7. Potential Optimizations . . . . . . . . . . . . . . . . . . . 28
7.1. Pipelining to Avoid Store-and-Forward Delays . . . . . . . 28
7.2. Deduplication . . . . . . . . . . . . . . . . . . . . . . 28
7.2.1. Traffic Deduplication . . . . . . . . . . . . . . . . 29
7.2.2. Cross-Server Storage Deduplication . . . . . . . . . . 30
8. Security Considerations . . . . . . . . . . . . . . . . . . . 30
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 30
10. Informative References . . . . . . . . . . . . . . . . . . . . 30
Appendix A. Appendix: Evaluation of Some Candidate Existing
Protocols for DECADE DRP and SDT . . . . . . . . . . 31
A.1. HTTP . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
A.1.1. HTTP Support for DECADE Resource Protocol A.1.1. HTTP Support for DECADE Resource Protocol
Primitives . . . . . . . . . . . . . . . . . . . . . . 24 Primitives . . . . . . . . . . . . . . . . . . . . . . 31
A.1.2. HTTP Support for DECADE Standard Transport A.1.2. HTTP Support for DECADE Standard Transport
Protocol Primitives . . . . . . . . . . . . . . . . . 24 Protocol Primitives . . . . . . . . . . . . . . . . . 32
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 25 A.1.3. Traffic Deduplication Primitives . . . . . . . . . . . 33
A.1.4. Other Operations . . . . . . . . . . . . . . . . . . . 33
A.1.5. Conclusions . . . . . . . . . . . . . . . . . . . . . 33
A.2. WEBDAV . . . . . . . . . . . . . . . . . . . . . . . . . . 33
A.2.1. WEBDAV Support for DECADE Resource Protocol
Primitives . . . . . . . . . . . . . . . . . . . . . . 34
A.2.2. WebDAV Support for DECADE Standard Transport
Protocol Primitives . . . . . . . . . . . . . . . . . 35
A.2.3. Other Operations . . . . . . . . . . . . . . . . . . . 35
A.2.4. Conclusions . . . . . . . . . . . . . . . . . . . . . 36
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 37
1. Introduction 1. Introduction
Peer-to-peer (P2P) applications have become widely used on the Peer-to-peer (P2P) applications have become widely used on the
Internet today to distribute contents, and they contribute a large Internet today to distribute contents, and they contribute a large
portion of the traffic in many networks. The DECADE Working Group portion of the traffic in many networks. The DECADE Working Group
has been formed with the goal of developing an architecture to has been formed with the goal of developing an architecture to
introduce in-network storage to be used by such applications, to introduce in-network storage to be used by such applications, to
achieve more efficient content distribution. Specifically, in many achieve more efficient content distribution. Specifically, in many
subscriber networks, it is typically more expensive to upgrade subscriber networks, it is typically more expensive to upgrade
skipping to change at page 5, line 50 skipping to change at page 5, line 50
alternative IETF protocols. alternative IETF protocols.
This document proceeds in two steps. First, it details the core This document proceeds in two steps. First, it details the core
architectural principles that can guide the DECADE design. Next, architectural principles that can guide the DECADE design. Next,
given these core principles, this document presents the core given these core principles, this document presents the core
components of the DECADE architecture and identifies usage of components of the DECADE architecture and identifies usage of
existing protocols and where there is a need for new protocol existing protocols and where there is a need for new protocol
development. development.
This document will be updated to track the progress of the DECADE This document will be updated to track the progress of the DECADE
survey [I-D.ietf-decade-survey] and requirements [I-D.gu-decade-reqs] survey [I-D.ietf-decade-survey] and requirements
drafts. [I-D.ietf-decade-reqs] drafts.
2. Entities 2. Entities
2.1. DECADE Storage Servers 2.1. DECADE Storage Servers
DECADE storage servers are operated by DECADE storage providers and DECADE storage servers are operated by DECADE storage providers and
provide the DECADE functionality as specified in this document, provide the DECADE functionality as specified in this document,
including mechanisms to store, retrieve and manage data. A storage including mechanisms to store, retrieve and manage data. A storage
provider may typically operate multiple storage servers. provider may typically operate multiple storage servers.
skipping to change at page 7, line 20 skipping to change at page 7, line 20
Content Consumer, or both. Content Consumer, or both.
An Application End-Point need not be an active member of a "swarm" to An Application End-Point need not be an active member of a "swarm" to
interact with the DECADE storage system. That is, an End-Point may interact with the DECADE storage system. That is, an End-Point may
interact with the DECADE storage servers as an offline activity. interact with the DECADE storage servers as an offline activity.
3. Architectural Principles 3. Architectural Principles
We identify the following key principles. We identify the following key principles.
3.1. Decoupled Control and Data Planes 3.1. Decoupled Control/Metadata and Data Planes
The DECADE infrastructure is intended to support multiple content The DECADE infrastructure is intended to support multiple content
distribution applications. A complete content distribution distribution applications. A complete content distribution
application implements a set of control functions including content application implements a set of control and management functions
search, indexing and collection, access control, ad insertion, including content search, indexing and collection, access control, ad
replication, request routing, and QoS scheduling. Different content insertion, replication, request routing, and QoS scheduling. A
distribution applications can have unique considerations designing observation of DECADE is that different content distribution
the control and signaling functions. For example, a major applications can have unique considerations designing the control and
competitive advantage of many successful P2P systems is their signaling functions:
substantial expertise in achieving highly efficient utilization of
peer and infrastructural resources. For instance, many live P2P o Metadata Management: Traditional file systems provide a standard
systems have their specific algorithms in constructing topologies to metadata abstraction: a recursive structure of directories to
achieve low-latency, high-bandwidth streaming. They continue to offer namespace management; each file is an opaque byte stream.
fine-tune such algorithms. In other words, in-network storage should In content distribution, applications may use different metadata
export basic mechanisms and allow as much flexibility as possible to management schemes. For example, one application may use a
the control planes to implement specific policies. This conforms to sequence of blocks (e.g., for file sharing), while another
the end-to-end systems principle and allows innovation and application may use a sequence of frames (with different sizes)
satisfaction of specific business goals. indexed by time. For example, Apple Live Streaming uses a dynamic
playlist to allow switching of frames encoded at different
encoding rates.
o Resource and Access Control: For example, a major competitive
advantage of many successful P2P systems is their substantial
expertise in achieving highly efficient utilization of peer and
infrastructural resources. For instance, many live P2P systems
have their specific algorithms in constructing topologies to
achieve low-latency, high-bandwidth streaming. They continue to
fine-tune such algorithms.
Given the diversity of control-plane functions, in-network storage
should export basic mechanisms and allow as much flexibility as
possible to the control planes to implement specific policies. This
conforms to the end-to-end systems principle and allows innovation
and satisfaction of specific business goals.
Specifically, in the DECADE architecture, the control plane focuses Specifically, in the DECADE architecture, the control plane focuses
on the application-specific, complex, and/or processing intensive on the application-specific, complex, and/or processing intensive
functions while the data plane provides storage and data transport functions while the data plane provides storage and data transport
functions. functions.
o Control plane: Signals details of where the data is to be o Control plane: Signals details of where the data is to be
downloaded from. The control signals may also include the time, downloaded from. The control signals may also include the time,
quality of service, and receivers of the download. It also quality of service, and receivers of the download. It also
provides higher layer meta-data management functions such as provides higher layer meta-data management functions such as
defining the sequence of data blocks forming a higher layer defining the sequence of data blocks forming a higher layer
content object. These are behaviors designed and implemented by content object. These are behaviors designed and implemented by
the Application. By Application, we mean the broad sense that the Application. By Application, we mean the broad sense that
includes other control plane protocols. includes other control plane protocols.
o Data plane: Stores and transfers data as instructed by the o Data plane: Stores and transfers basic data objects as instructed
Application's Control Plane. by the Application's Control Plane.
Decoupling control plane and data plane is not new. For example, Decoupling control plane and data plane is not new. For example,
OpenFlow is an implementation of this principle for Internet routing, OpenFlow is an implementation of this principle for Internet routing,
where the computation of the forwarding table and the application of where the computation of the forwarding table and the application of
the forwarding table are separated. Google File System applies the the forwarding table are separated. Google File System applies the
principle to file system design, by utilizing the Master to handle principle to file system design, by utilizing the Master to handle
the meta-data management, and the Chunk Servers to handle the data the meta-data management, and the Chunk Servers to handle the data
plane functions (i.e., read and write of chunks of data). NFS4 also plane functions (i.e., read and write of chunks of data). NFS4 also
implements this principle. implements this principle.
skipping to change at page 12, line 7 skipping to change at page 12, line 7
The primary focus of the current version of this document is on the The primary focus of the current version of this document is on the
architectural principles. The detailed system components will be architectural principles. The detailed system components will be
discussed in the next document revision. discussed in the next document revision.
This section presents an overview of the components in the DECADE This section presents an overview of the components in the DECADE
architecture. architecture.
.--------------------------------------------------------------. .--------------------------------------------------------------.
| Application End-Point | | Application End-Point |
| .------------. .-----------------. .----------. | | .------------. .-------------------. |
| | App-Layer | ... | App Data Object | | App Data | | | | App-Layer | ... | App Data Assembly | |
| | Algorithms | | Sequencing | | Naming | | | | Algorithms | | Sequencing | |
| `------------' `-----------------' `----------' | | `------------' `-------------------' |
| | | |
| .----------------------------------------------------------. | | .----------------------------------------------------------. |
| | DECADE Client | | | | DECADE Client | |
| | | | | | | |
| | .-------------------------. .--------------------------. | | | | .-------------------------. .--------------------------. | |
| | | Resource Controller | | Data Controller | | | | | | Resource Controller | | Data Controller | | |
| | | .--------. .----------. | | .------------. .-------. | | | | | | .--------. .----------. | | .------------. .-------. | | |
Native | | | | Data | | Resource | | | | Data | | Data | | | | Native | | | | Data | | Resource | | | | Data | | Data | | | |
App | | | | Access | | Sharing | | | | Scheduling | | Index | | | | App | | | | Access | | Sharing | | | | Scheduling | | Index | | | |
Protocol(s)| | | | Policy | | Policy | | | | | | | | | | Protocol(s)| | | | Policy | | Policy | | | | | | | | | |
skipping to change at page 13, line 27 skipping to change at page 13, line 27
4.1. Content Distribution Application 4.1. Content Distribution Application
Content Distribution Applications have many functional components. Content Distribution Applications have many functional components.
For example, many P2P applications have components to manage overlay For example, many P2P applications have components to manage overlay
topology management, piece selection, etc. In supporting DECADE, it topology management, piece selection, etc. In supporting DECADE, it
may be advantageous to consider DECADE within some of these may be advantageous to consider DECADE within some of these
components. However, in this architecture document, we focus on the components. However, in this architecture document, we focus on the
components directly employed to support DECADE. components directly employed to support DECADE.
4.1.1. Data Sequencing and Naming 4.1.1. Data Assembly
DECADE is primarily designed to support applications that can divide DECADE is primarily designed to support applications that can divide
distributed contents into immutable data objects. To accomplish distributed contents into immutable data objects. To accomplish
this, applications include a component responsible for re-assembling this, applications include a component responsible for creating the
data objects and also creating the individual data objects. We call individual data objects before distribution and then re-assembling
this component Application Data Sequencing. The specific data objects at the Content Consumer. We call this component
implementation is entirely decided by the application. Application Data Assembly. The specific implementation is entirely
decided by the application.
In assembling or producing the data objects, an important In producing and assembling the data objects, two important
consideration is the naming of these objects. We call the component considerations are sequencing and naming. The DECADE architecture
responsible for assigning and interpreting application-layer names assumes that applications implement this functionality themselves.
the Application Data Naming component. The specific implementation
is entirely decided by the application. For example, a Content Distribution Application might divide a single
content (e.g., a finite-length file or a live stream) into multiple
data objects with names of the form "CONTENT_ID:SEQUENCE_NUMBER"
where CONTENT_ID identifies the particular content (e.g., a
particular movie or TV channel distributed by the application), and
SEQUENCE_NUMBER both identifies an individual data object and
determines its order when a client reconstructs individual data
objects into the full content.
4.1.2. Native Protocols 4.1.2. Native Protocols
Applications may still use existing protocols. In particular, an Applications may still use existing protocols. In particular, an
application may reuse existing protocols primarily for control/ application may reuse existing protocols primarily for control/
signaling. However, an application may still retain its existing signaling. However, an application may still retain its existing
data transport protocols, in addition to DECADE as the data transport data transport protocols, in addition to DECADE as the data transport
protocol. This can be important for applications that are designed protocol. This can be important for applications that are designed
to be highly robust (e.g., if DECADE servers are unavailable). to be highly robust (e.g., if DECADE servers are unavailable).
skipping to change at page 16, line 11 skipping to change at page 16, line 20
The DECADE architecture specification will provide exactly one DECADE The DECADE architecture specification will provide exactly one DECADE
Resource Protocol. Resource Protocol.
4.3.2. Standard Data Transports 4.3.2. Standard Data Transports
Existing data transport protocols are used to read and write data Existing data transport protocols are used to read and write data
from a DECADE Server. Protocols under consideration are WebDAV and from a DECADE Server. Protocols under consideration are WebDAV and
NFS. NFS.
4.4. DECADE Data Sequencing and Naming 4.4. Data Sequencing and Naming
We have discussed above that an Application may have its own behavior
for both sequencing and naming data objects. In order to provide a
simple and generic interface, the DECADE Server is only responsible
for storing and retrieving individual data objects.
DECADE names are not necessarily correlated with the naming or In order to provide a simple and generic interface, the DECADE Server
sequencing used by the Application using a DECADE client. The DECADE is only responsible for storing and retrieving individual data
client is expected to maintain a mapping from its own naming to the objects. Furthermore, DECADE uses its own simple naming scheme that
DECADE naming. Furthermore, the DECADE naming scheme implies no provides uniqueness (with high probability) between data objects,
sequencing or grouping of objects, even if this is done at the even across multiple applications.
application layer.
Multiple applications may make use of a DECADE infrastructure, and 4.4.1. DECADE Data Object Naming Schame
each Application may employ its own naming scheme. To remain
independent of particular applications, DECADE uses a simple, common
naming scheme that supports unique naming of individual data objects.
This is achieved by deriving object names from hashes of the object
content. This scheme is made possible by the fact that DECADE data
objects are immutable. Details of the naming scheme (complete
syntax, hash algorithms etc.) will be defined in a future version of
this document.
By naming data objects directly as the content hash, DECADE names The name of a data object is derived from the hash over the data
satisfy important objectives: object's content (the raw bytes), which is made possible by the fact
that DECADE objects are immutable. This scheme multiple appealing
properties:
o Simple integrity verification o Simple integrity verification
o Unique names (with high probability) o Unique names (with high probability)
o Application independent, without a new IANA-maintained registry o Application independent, without a new IANA-maintained registry
A particular advantage of using the content hash is that it is The DECADE naming scheme also includes a "type" field, the "type"
straightforward for as server or client to validate a data object identifier indicates that the name is the hash of the data object's
before storing or transmitting it. While these capabilities could be content and the particular hashing algorithm used. This allows the
achieved by supplying the content hash in both read and write DECADE protocol to evolve by either changing the hashing algorithm
requests as metadata, using the content hash as the name satisfies (e.g., if security vulernabilities with an existing hashing algorithm
the objectives and is straightforward. are discovered), or move to a different naming scheme altogether.
The specific format of the name (e.g., encoding, hash algorithms,
etc) is out of scope of this document, and left for protocol
specification.
Another advantage of this scheme is that a DECADE client knows the Another advantage of this scheme is that a DECADE client knows the
name of a data object before it is completely stored at the DECADE name of a data object before it is completely stored at the DECADE
server. This allows for particular optimizations, such as server. This allows for particular optimizations, such as
advertising data object while the data object is being stored, advertising data object while the data object is being stored,
removing store-and-forward delays. For example, a DECADE client A removing store-and-forward delays. For example, a DECADE client A
may simultaneously begin storing an object to a DECADE server, and may simultaneously begin storing an object to a DECADE server, and
advertise that the object is available to DECADE client B. If it is advertise that the object is available to DECADE client B. If it is
supported by the DECADE server, client B may begin downloading the supported by the DECADE server, client B may begin downloading the
object before A is finished storing the object. object before A is finished storing the object.
4.5. In-Network Storage Components Mapped to DECADE Architecture 4.4.2. Application Usage
Recall from Section 4.1.1 that an Application typically includes its
own naming and sequencing scheme. It is important to note that the
DECADE naming format does not attempt to replace any naming or
sequencing of data objects already performed by an Application;
instead, the DECADE naming is intended to apply only to data objects
referenced at the DECADE layer.
DECADE names are not necessarily correlated with the naming or
sequencing used by the Application using a DECADE client. The DECADE
client is expected to maintain a mapping from its own data objects
and their names to the DECADE data objects and names. Furthermore,
the DECADE naming scheme implies no sequencing or grouping of
objects, even if this is done at the application layer.
Not only does an Application retain its own naming scheme, it may
also decide the sizes of data objects to be distributed via DECADE.
This is desirable since sizes of data objects may impact Application
performance (e.g., overhead vs. data distribution delay), and the
particular tradeoff is application-dependent.
4.4.3. Application Usage Example
To illustrate these properties, this section presents multiple
examples.
4.4.3.1. Application with Fixed-Size Chunks
Similar to the example in Section 4.1.1, consider an Application in
which each individual application-layer segment of data is called a
"chunk" and has a name of the form: "CONTENT_ID:SEQUENCE_NUMBER".
Furthermore, assume that the application's native protocol uses
chunks of size 16KB.
Now, assume that this application wishes to make use of DECADE, and
assume that it wishes to store data to DECADE servers in data objects
of size 64KB. To accomplish this, it can map a sequence of 4 chunks
into a single DECADE object, as shown in Figure 2.
Application Chunks
.---------.---------.---------.---------.---------.---------.---------.--
| | | | | | | |
| Chunk_0 | Chunk_1 | Chunk_2 | Chunk_3 | Chunk_4 | Chunk_5 | Chunk_6 |
| | | | | | | |
`---------`---------`---------`---------`---------`---------`---------`--
DECADE Data Objects
.---------------------------------------.--------------------------------
| |
| Object_0 | Object_1
| |
`---------------------------------------`--------------------------------
Figure 2: Mapping Application Chunks to DECADE Data Objects
In this example, the Application might maintain a logical mapping
that is able to determine the name of a DECADE data object given the
chunks contained within that data object. The name might be learned
from either the original source, another endpoint with which the it
is communicating, a tracker, etc.
It is important to note that as long as the data contained within
each sequence of chunks is unique, the corresponding DECADE data
objects have unique names. This is desired, and happens
automatically if particular Application segments the same stream of
data in a different way, including different chunk size sizes or
different padding schemes.
4.4.3.2. Application with Continuous Streaming Data
Next, consider an Application whose native protocol retrieves a
continuous data stream (e.g., an MPEG2 stream) instead of downloading
and redistributing chunks of data. Such an application could segment
the continuous data stream to produce either fixed-sized or variable-
sized DECADE data objects.
Figure 3 shows how a video streaming application might produce
variable-sized DECADE data objects such that each DECADE data object
contains 10 seconds of video data.
Application's Video Stream
.------------------------------------------------------------------------
|
|
|
`------------------------------------------------------------------------
^ ^ ^ ^ ^
| | | | |
0 Seconds 10 Seconds 20 Seconds 30 Seconds 40 Seconds
0 B 400 KB 900 KB 1200 KB 1500 KB
DECADE Data Objects
.--------------.--------------.--------------.--------------.------------
| | | | |
| Object_0 | Object_1 | Object_2 | Object_3 |
| (400 KB) | (500 KB) | (300 KB) | (300 KB) |
`--------------`--------------`--------------`--------------`------------
Figure 3: Mapping a Continuous Data Stream to DECADE Data Objects
Similar to the previous example, the Application might maintain a
mapping that is able to determine the name of a DECADE data object
given the time offset of the video chunk.
4.5. Token-based Authentication and Resource Control
A primary use case for DECADE is a DECADE Client authorizing other
DECADE Clients to store or retrieve data objects from its DECADE
storage. To support this, DECADE uses a token-based authentication
scheme.
In particular, an entity trusted by a DECADE Client generates a
digitally-signed token with particular properties (see Section 5.1.2
for details). The DECADE Client distributes this token to other
DECADE Clients which then use the token when sending requests to the
DECADE Server. Upon receiving a token, the DECADE Server validates
the signature and the operation being performed.
This is a simple scheme, but has multiple important advantages over
an alternate approach in which a DECADE Client explicitly manipulates
an Access Control List (ACL) associated with each DECADE data object.
In particular, it has the following advantages when applied to
DECADE's target applications:
o Authorization policies are implemented within the Application; an
Application explicitly controls when tokens are generated and to
whom they are distributed.
o Fine-grained access and resource control can be applied to data
objects; see Section 5.1.2 for the list of restrictions that can
be enforced with a token.
o There is no messaging between a DECADE Client and DECADE Server to
manipulate data object permissions. This can simplify, in
particular, Applications which share data objects with many
dynamic peers and need to frequently adjust access control
policies attached to DECADE data objects.
o Tokens can provide anonymous access, in which a DECADE Server does
not need to know the identity of each DECADE Client that accesses
it. This enables a DECADE Client to send tokens to DECADE Clients
in other administrative or security domains, and allow them to
read or write data objects from its DECADE storage.
It is important to note that, in addition to DECADE Clients applying
access control policies to DECADE data objects, the DECADE Server may
be configured to apply additional policies based on user, object,
geographic location, etc. Defining such policies is out of scope of
the DECADE Working Group, but in such a case, a DECADE Client may be
denied access even though it possess a valid token.
4.6. In-Network Storage Components Mapped to DECADE Architecture
In this section we evaluate how the basic components of an in-network In this section we evaluate how the basic components of an in-network
storage system identified in Section 3 of [I-D.ietf-decade-survey] storage system identified in Section 3 of [I-D.ietf-decade-survey]
map into the DECADE architecture. map into the DECADE architecture.
It is important to note that complex and/or application-specific It is important to note that complex and/or application-specific
behavior is delegated to applications instead of tuning the storage behavior is delegated to applications instead of tuning the storage
system wherever possible. system wherever possible.
4.5.1. Data Access Interface 4.6.1. Data Access Interface
Users can read and write objects of arbitrary size through the DECADE Users can read and write objects of arbitrary size through the DECADE
Client's Data Controller, making use of a standard data transport. Client's Data Controller, making use of a standard data transport.
4.5.2. Data Management Operations 4.6.2. Data Management Operations
Users can move or delete previously stored objects via the DECADE Users can move or delete previously stored objects via the DECADE
Client's Data Controller, making use of a standard data transport. Client's Data Controller, making use of a standard data transport.
4.5.3. Data Search Capability 4.6.3. Data Search Capability
Users can enumerate or search contents of DECADE servers to find Users can enumerate or search contents of DECADE servers to find
objects matching desired criteria through services provided by the objects matching desired criteria through services provided by the
Content Distribution Application (e.g., buffer-map exchanges, a DHT, Content Distribution Application (e.g., buffer-map exchanges, a DHT,
or peer-exchange). In doing so, End-Points may consult their local or peer-exchange). In doing so, End-Points may consult their local
data index in the DECADE Client's Data Controller. data index in the DECADE Client's Data Controller.
4.5.4. Access Control Authorization 4.6.4. Access Control Authorization
All methods of access control are supported: public-unrestricted, All methods of access control are supported: public-unrestricted,
public-restricted and private. Access Control Policies are generated public-restricted and private. Access Control Policies are generated
by a Content Distribution Application and provided to the DECADE by a Content Distribution Application and provided to the DECADE
Client's Resource Controller. The DECADE Server is responsible for Client's Resource Controller. The DECADE Server is responsible for
implementing the access control checks. implementing the access control checks.
4.5.5. Resource Control Interface 4.6.5. Resource Control Interface
Users can manage the resources (e.g. bandwidth) on the DECADE server Users can manage the resources (e.g. bandwidth) on the DECADE server
that can be used by other Application End-Points. Resource Sharing that can be used by other Application End-Points. Resource Sharing
Policies are generated by a Content Distribution Application and Policies are generated by a Content Distribution Application and
provided to the DECADE Client's Resource Controller. The DECADE provided to the DECADE Client's Resource Controller. The DECADE
Server is responsible for implementing the resource sharing policies. Server is responsible for implementing the resource sharing policies.
4.5.6. Discovery Mechanism 4.6.6. Discovery Mechanism
This is outside the scope of the DECADE architecture. However, it is This is outside the scope of the DECADE architecture. However, it is
expected that DNS or some other well known protocol will be used for expected that DNS or some other well known protocol will be used for
the users to discover the DECADE servers. the users to discover the DECADE servers.
4.5.7. Storage Mode 4.6.7. Storage Mode
DECADE Servers provide an object-based storage mode. Immutable data DECADE Servers provide an object-based storage mode. Immutable data
objects may be stored at a DECADE server. Applications may consider objects may be stored at a DECADE server. Applications may consider
existing blocks as DECADE data objects, or they may adjust block existing blocks as DECADE data objects, or they may adjust block
sizes before storing in a DECADE server. sizes before storing in a DECADE server.
5. DECADE Protocols 5. DECADE Protocols
This section specifies the DECADE Resource Protocol (DRP) and the This section specifies the DECADE Resource Protocol (DRP) and the
Standard Data Transport (SDT) in terms of abstract protocol Standard Data Transport (SDT) in terms of abstract protocol
interactions that are intended to mapped to specific protocols such interactions that are intended to mapped to specific protocols. Note
as HTTP. It is possible that a single specific protocol provides that while the protocols are logically separate, DRP is specified as
both, DRP and SDT functionality. being carried through extension fields within an SDT (e.g., HTTP
headers).
The DRP is the protocol used by a DECADE client to configure the The DRP is the protocol used by a DECADE client to configure the
resources and authorization used to satisfy requests (reading, resources and authorization used to satisfy requests (reading,
writing, and management operations concerning DECADE objects) at a writing, and management operations concerning DECADE objects) at a
DECADE server. The SDT is used to send the operations to the DECADE DECADE server. The SDT is used to send the operations to the DECADE
server. Necessary DRP metadata is supplied using mechanisms in the server. Necessary DRP metadata is supplied using mechanisms in the
SDT that are provided for extensibility (e.g., additional request SDT that are provided for extensibility (e.g., additional request
parameters). A detailed architectural description of the DRP and SDT parameters or extension headers).
is still a work in progress. Important aspects, including details of
authorization, will be added in a future version of this document.
5.1. DECADE Resource Protocol (DRP) 5.1. DECADE Resource Protocol (DRP)
DRP provides configuration of access control and resource sharing DRP provides configuration of access control and resource sharing
policies on DECADE servers. A content distribution application, policies on DECADE servers. A content distribution application,
e.g., a live P2P streaming session, MAY employ several DECADE e.g., a live P2P streaming session, MAY employ several DECADE
servers, for instance, servers in different operator domains, and DRP servers, for instance, servers in different operator domains, and DRP
allows one instance of such an application, e.g., an application allows one instance of such an application, e.g., an application
endpoint, to configure access control and resource sharing policies endpoint, to configure access control and resource sharing policies
on a set of servers. on a set of servers.
On a single DECADE server, the following resources have to be 5.1.1. Controlled Resources
managed:
communication resources: DECADE servers may limited communication On a single DECADE server, the following resources may be managed:
communication resources: DECADE servers have limited communication
resources in terms of bandwidth (upload/download) but also in resources in terms of bandwidth (upload/download) but also in
terms of number of connected clients (connections) at a time. terms of number of connected clients (connections) at a time.
storage resources: DECADE servers may limited storage resources. storage resources: DECADE servers have limited storage resources.
Note: this list of resources will be extended in a future version of 5.1.2. Token-based Authentication and Resource Control
this document.
DECADE uses a token-based scheme that allows a DECADE Client to
authorize other DECADE Clients to perform certain actions (e.g., read
or write data objects) on the client's DECADE Server. The token
includes the following fields:
Permitted operations (e.g., read, write)
Permitted objects (e.g., names of data objects that may be read or
written)
Permitted clients (e.g., as indicated by IP address or other
identifier) that may use the token
Expiration time
Priority for bandwidth given to requested operation
Amount of data that may be read or written
The particular format for the token is out of scope of this document.
The tokens are generated by a trusted entity at the request of a
DECADE Client. It is out of scope of this document to identify which
entity serves this purpose, but examples include the DECADE Client
itself, a DECADE Server trusted by the DECADE Client, or another
server managed by a Storage Provider trusted by the DECADE Client.
Upon generating a token, a DECADE Client may distribute it to another
DECADE Client (e.g., via their native Application protocol). The
receiving DECADE Client may then connect to the sending DECADE
Client's DECADE Server and perform any operation permitted by the
token. The token must be sent along with the operation. The DECADE
Server validates the token to identify the DECADE Client that issued
it and whether the requested operation is permitted by the contents
of the token. If the token is successfully validated, the DECADE
Server applies the resource control policies indicated in the token
while performing the operation.
It is possible for DRP to allow tokens to apply to a batch of
operations to reduce communication overhead required between DECADE
Clients.
DRP may also define tokens to include a unique identifer to allow a
DECADE Server to detect when a token is used multiple times.
5.1.3. Status Information
DRP provides a request service for status information that DECADE
clients can use to request information from a DECADE server.
status information per application context on a specific server:
Access to such status information requires client authorization,
i.e., DECADE clients need to be authorized to access status
information for a specific application context. This
authorization (and the mapping to application contexts) is based
on the user delegation concept as described in Section 3.5. The
following status information elements can be obtained:
* list of associated objects (with properties)
* resources used/available
* list of servers to which objects have been distributed (in a
certain time-frame)
* list of clients to which objects have been distributed (in a
certain time-frame)
For the list of servers/clients to which objects have been
distributed to, the DECADE server can decide on time bounds for
which this information is stored and specify the corresponding
time frame in the response to such requests. Some of this
information can be used for accounting purposes, e.g., the list of
clients to which objects have been distributed.
access information per application context on a specific server:
Access information can be provided for accounting purposes, for
example, when application service providers are interested to
maintain access statistics for resources and/or to perform
accounting per user. Again, access to such information requires
client authorization based on the user delegation concept as
described in Section 3.5. The following access information
elements can be requested:
* what objects have been accessed how many times
* access tokens that a server as seen for a given object
The DECADE server can decide on time bounds for which this
information is stored and specify the corresponding time frame in
the response to such requests.
5.1.4. Object Properties
Objects that are stored on a DECADE server can provide properties (in
addition to the object identifier and the actual content). Depending
on authorization, DECADE clients may get or set such properties.
This authorization (and the mapping to application contexts) is based
on the user delegation concept as described in Section 3.5. The
DECADE architecture does not limit the set of permissible properties,
but rather specifies a set of baseline properties that SHOULD be
supported by implementations.
TTL: TTL of the object as an absolute time value
object size: in bytes
MIME type
access statistics: how often the object has been accessed (and what
tokens have been used)
5.2. Standard Data Transport (SDT) 5.2. Standard Data Transport (SDT)
A DECADE server provide a data access interface, and SDT is used to A DECADE server provide a data access interface, and SDT is used to
write objects to a server and to read (download) objects from a write objects to a server and to read (download) objects from a
server. Semantically, SDT is a client-server protocol, i.e., the server. Semantically, SDT is a client-server protocol, i.e., the
DECADE server always responds to client requests. DECADE server always responds to client requests.
5.2.1. Writing/Uploading Objects 5.2.1. Writing/Uploading Objects
skipping to change at page 22, line 37 skipping to change at page 28, line 25
time). time).
In the case of a GET operation, the DECADE server is to retrieve the In the case of a GET operation, the DECADE server is to retrieve the
data object from the remote server using the specified credentials data object from the remote server using the specified credentials
(via a GET request to the remote server), and then return the object (via a GET request to the remote server), and then return the object
to the client. In the case of a PUT operation, the DECADE server is to the client. In the case of a PUT operation, the DECADE server is
to store the object from the client, and then store the object to the to store the object from the client, and then store the object to the
remote server using the specified credentials (via a PUT request to remote server using the specified credentials (via a PUT request to
the remote server). the remote server).
6.2. Potential Optimizations 7. Potential Optimizations
As a suggestion to the protocol and eventual implementations, we As suggestions for the protocol design and eventual implementations,
would like to point out particular optimizations that could be made we discuss particular optimizations that are enabled by the DECADE
when making use of the server-to-server protocol. Architecture discussed in this document.
6.2.1. Pipelining to Avoid Store-and-Forward Delays 7.1. Pipelining to Avoid Store-and-Forward Delays
A DECADE server may choose to not fully store an object before DECADE server may choose to not fully store an object before
beginning to serve it. For example, in the case of a GET request, a beginning to serve it. For example, in the case of a GET request, a
DECADE server may begin to receive a data object from a remote DECADE server may begin to receive a data object from a remote server
server, and immediately begin returning it to the DECADE client. or DECADE Client, and immediately begin returning it to the DECADE
This pipelining mode avoids store-and-forward delays, which could be client. This pipelining mode avoids store-and-forward delays, which
substantial for large objects. A similar behavior could be used for could be substantial for large objects. A similar behavior could be
PUT. used for PUT.
7. Security Considerations 7.2. Deduplication
A common concern amongst Storage Providers is the total volume of
data that needs to be stored. An optimization frequently applied in
existing storage systems is de-duplication techniques which attempt
to avoid storing identical data multiple times. DECADE Server
implementations may internally perform de-duplication of data on
disk, but the DECADE architecture enables other forms of de-
duplication.
Note that these techniques may impact protocol design. Discussion of
whether or not they should be adopted is out of scope of this
document.
7.2.1. Traffic Deduplication
7.2.1.1. Rationale
When a DECADE client (A) indicates its DECADE account on a DECADE
server (S) to fetch an object from a remote entity (R) (a DECADE
server or DECADE client) and if the object is already stored locally
in S, S may perform Traffic Deduplication. This means that S does
not download the object from R, which saves network traffic.
Instead, it performs a challenge to make sure that the remote entity
R actually has the object and then replies with its local object copy
directly.
7.2.1.2. Example
As shown in Figure 4 , without Traffic Deduplication, redundant
traffic flows between S and R will be issued if the server already
has the object requested by A. If Traffic Deduplication is enabled, S
only needs to challenge R to verify that it does have the data to
avoid data-stealing attacks.
A S R
+----------+ obj req +------------+ obj req +----------+
| DECADE |=========>| A's |==========>| Remote |
| CLIENT |<=========| Account |<==========| Entity |
+----------+ obj rsp +------------+ obj rsp +----------+
(a) Without Traffic Deduplication
A S R
+----------+ obj req +------------+ challenge +----------+
| DECADE |=========>| A's |---------->| Remote |
| CLIENT |<=========| Account |<----------| Entity |
+----------+ obj rsp +------------+ obj hash +----------+
(b) With Traffic Deduplication
Figure 4
7.2.1.3. HTTP Compatibility of Challenge
How to integrate traffic deduplication with HTTP is shown in
Appendix A.1.3.
7.2.2. Cross-Server Storage Deduplication
The same object might be uploaded for multiple times to different
DECADE servers. For storage efficiency, storage providers may desire
a single object to be stored on one or a few servers. They might
design internal system architecture to achieve that, or simply
redirect the requests to proper servers. DECADE protocol support
redirections of DECADE client request to support further cross-server
storage deduplication.
8. Security Considerations
This document currently does not contain any security considerations This document currently does not contain any security considerations
beyond those mentioned in [I-D.ietf-decade-problem-statement]. beyond those mentioned in [I-D.ietf-decade-problem-statement].
8. IANA Considerations 9. IANA Considerations
This document does not have any IANA considerations. This document does not have any IANA considerations.
9. Informative References 10. Informative References
[RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,
Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext
Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.
[RFC3744] Clemm, G., Reschke, J., Sedlar, E., and J. Whitehead, "Web
Distributed Authoring and Versioning (WebDAV)
Access Control Protocol", RFC 3744, May 2004.
[RFC4331] Korver, B. and L. Dusseault, "Quota and Size Properties fo
r Distributed Authoring and Versioning (DAV) Collections",
RFC 4331, February 2006.
[RFC4709] Reschke, J., "Mounting Web Distributed Authoring and
Versioning (WebDAV) Servers", RFC 4709, October 2006.
[RFC4918] Dusseault, L., "HTTP Extensions for Web Distributed
Authoring and Versioning (WebDAV)", RFC 4918, June 2007.
[I-D.ietf-decade-problem-statement] [I-D.ietf-decade-problem-statement]
Song, H., Zong, N., Yang, Y., and R. Alimi, "DECoupled Song, H., Zong, N., Yang, Y., and R. Alimi, "DECoupled
Application Data Enroute (DECADE) Problem Statement", Application Data Enroute (DECADE) Problem Statement",
draft-ietf-decade-problem-statement-02 (work in progress), draft-ietf-decade-problem-statement-03 (work in progress),
January 2011. March 2011.
[I-D.ietf-decade-survey] [I-D.ietf-decade-survey]
Alimi, R., Rahman, A., and Y. Yang, "A Survey of In- Alimi, R., Rahman, A., and Y. Yang, "A Survey of In-
network Storage Systems", draft-ietf-decade-survey-04 network Storage Systems", draft-ietf-decade-survey-04
(work in progress), March 2011. (work in progress), March 2011.
[I-D.gu-decade-reqs] [I-D.ietf-decade-reqs]
Yingjie, G., Bryan, D., Yang, Y., and R. Alimi, "DECADE Yingjie, G., Bryan, D., Yang, Y., and R. Alimi, "DECADE
Requirements", draft-gu-decade-reqs-05 (work in progress), Requirements", draft-ietf-decade-reqs-02 (work in
July 2010. progress), May 2011.
Appendix A. Appendix: Evaluation of Candidate Existing Protocols for [GoogleStorageDevGuide]
DECADE DRP and SDT "Google Storage Developer Guide", <http://code.google.com/
apis/storage/docs/developer-guide.html>.
In this section we illustrate how the abstract protocol interactions Appendix A. Appendix: Evaluation of Some Candidate Existing Protocols
specified in Section 5 for DECADE DRP and SDT can be fulfilled by for DECADE DRP and SDT
existing protocols such as HTTP and WEBDAV. (This version of the
document considers HTTP only, a future version will provide a In this section we evaluate how well the abstract protocol
possible WEBDAV-based illustration as well.) interactions specified in Section 5 for DECADE DRP and SDT can be
fulfilled by existing protocols such as HTTP and WEBDAV.
A.1. HTTP A.1. HTTP
HTTP is a key protocol for the Internet and specifically the World HTTP [RFC2616] is a key protocol for the Internet in general and
Wide Web. HTTP is a request-response protocol. A typical transaction especially for the World Wide Web. HTTP is a request-response
is when a client (e.g. web browser) requests content (resources) from protocol. A typical transaction involves a client (e.g. web browser)
a web server. Other examples are when the client puts or deletes requesting content (resources) from a web server. Another example is
content from the server. when a client stores or deletes content from a server.
A.1.1. HTTP Support for DECADE Resource Protocol Primitives A.1.1. HTTP Support for DECADE Resource Protocol Primitives
DRP provides configuration of access control and resource sharing DRP provides configuration of access control and resource sharing
policies on DECADE servers. policies on DECADE servers.
A.1.1.1. Access Control Primitives A.1.1.1. Access Control Primitives
Access control requires mechanisms for defining the access policies Access control requires mechanisms for defining the access policies
for the server, and then checking the authorization of a user before for the server, and then checking the authorization of a user before
it stores or retrieves content. HTTP supports access control via it stores or retrieves content. HTTP supports a rudimentary access
"HTTP Secure" (HTTPS). HTTPS is a combination of HTTP with SSL/TLS. control via "HTTP Secure" (HTTPS). HTTPS is a combination of HTTP
The main purpose of HTTPS is to authenticate the server and encrypt with SSL/TLS. The main use of HTTPS is to authenticate the server
all traffic between the client and the server. HTTPS does not and encrypt all traffic between the client and the server. There is
support authentication of the client. also a mode to support client authentication though this is less
frequently used.
A.1.1.2. Communication Resource Controls Primitives A.1.1.2. Communication Resource Controls Primitives
Communications resources include bandwidth (upload/download) and Communications resources include bandwidth (upload/download) and
number of simultaneous connected clients (connections). HTTP number of simultaneous connected clients (connections). HTTP
supports communication resource control through "persistent" HTTP supports bandwidth control indirectly through "persistent" HTTP
connections. Persistent HTTP connections allows a client to keep connections. Persistent HTTP connections allows a client to keep
open the underlying TCP connection to the server to allow streaming open the underlying TCP connection to the server to allow streaming
and pipelining (multiple simultaneous requests). HTTP does not and pipelining (multiple simultaneous requests for a given client).
support a mechanism to allow limiting the communciation resources to
a client. HTTP does not define protocol operation to allow limiting the
communciation resources to a client. However servers typically
perform this function via implementation algorithms.
A.1.1.3. Storage Resource Control Primitives A.1.1.3. Storage Resource Control Primitives
Storage resources include amount of memory and lifetime of storage. Storage resources include amount of memory and lifetime of storage.
HTTP does not allow direct control of storage at the server end HTTP does not allow direct control of storage at the server end
point. However HTTP supports caching at intermediate points such as point. However HTTP supports caching at intermediate points such as
a web proxy. For this purpose, HTTP defines cache control mechanisms a web proxy. For this purpose, HTTP defines cache control mechanisms
that define how long and in what situations the intermediate point that define how long and in what situations the intermediate point
may store and use the content. may store and use the content.
A.1.2. HTTP Support for DECADE Standard Transport Protocol Primitives A.1.2. HTTP Support for DECADE Standard Transport Protocol Primitives
SDT is used to write objects and read (download) objects from a SDT is used to write objects and read (download) objects from a
DECADE server. The object can be either a self-contained object such DECADE server. The object can be either a self-contained object such
as a multimedia file or a chunk from a P2P system. as a multimedia file or a chunk from a P2P system.
A.1.2.1. Writing Primitives A.1.2.1. Writing Primitives
Writing involves uploading objects to the server. HTTP supports two Writing involves uploading objects to the server. HTTP supports two
methods of writing called PUT and POST. In HTTP the object is called methods of writing called PUT and POST. In HTTP the object is called
a resource and can be identified by a URL. PUT uploads a resource to a resource and is identified by a URI. PUT uploads a resource to a
a specific location on the server. POST, on the other hand, submits specific location on the server. POST, on the other hand, submits
the object to the server and the server decides whether to update an the object to the server and the server decides whether to update an
existing resource or to create a new resource. existing resource or to create a new resource.
For DECADE, the choice of whether to use PUT or POST will be
influenced by which entity is responsible for the naming. If the
client performs the naming, then PUT is appropriate. If the server
performs the naming, then POST should be used (to allow the server to
define the URI).
A.1.2.2. Downloading Primitives A.1.2.2. Downloading Primitives
Downloading involves fetching of an object from the server. HTTP Downloading involves fetching of an object from the server. HTTP
supports downloading through the GET method. GET fetches a specific supports downloading through the GET and HEAD methods. GET fetches a
resource as identified by the URL. specific resource as identified by the URL. HEAD is similiar but
only fetches the metadata ("header") associated with the resource but
not the resource itself.
A.1.2.3. Other Methods A.1.3. Traffic Deduplication Primitives
To challenge a remote entity for an object, the DECADE server should
provide a seed number, which is generated by the server randomly, and
ask the remote entity to return a hash calculated from the seed
number and the content of the object. The server MAY also specify
the hash function which the remote entity should use. HTTP supports
the challenge message through the GET methods. The message type
("challenge"), the seed number and the hash funtion name are put in
URL. In the reply, the hash is sent in an ETAG header.
A.1.4. Other Operations
HTTP supports deleting of content on the server through the DELETE HTTP supports deleting of content on the server through the DELETE
method. method.
A.1.5. Conclusions
HTTP can provide a rudimentary DRP and SDT for some aspects of
DECADE, but will not be able to satisfy all the DECADE requirements.
For example, HTTP does not provide a complete access control
mechanism, nor does it support storage resource controls at the end
point server.
It is possible, however, to envision combining HTTP with a custom
suite of other protocols to fulfill most of the DECADE requirements
for DRP and SDT. For example, Google Storage for Developers is built
using HTTP (with extensive proprietary extensions such as custom HTTP
headers). Google Storage also uses OAUTH 2.0 (for access control) in
combination with HTTP [GoogleStorageDevGuide].
A.2. WEBDAV
WebDAV [RFC4918] is a protocol for enhanced Web content creation and
management. It was developed as an extension to HTTP Appendix A.1.
WebDAV supports traditional operations for reading/writing from
storage, as well as more advanced features such as locking and
namespace management which are important when multiple users
collaborate to author or edit a set of documents. HTTP is a subset
of WebDAV functionality. Therefore, all the points noted above in
Appendix A.1 apply implicitly to WebDAV as well.
A.2.1. WEBDAV Support for DECADE Resource Protocol Primitives
DRP provides configuration of access control and resource sharing
policies on DECADE servers.
A.2.1.1. Access Control Primitives
Access control requires mechanisms for defining the access policies
for the server, and then checking the authorization of a user before
it stores or retrieves content. WebDAV has an Access Control
Protocol defined in [RFC3744].
The goal of WebDAV access control is to provide an interoperable
mechanism for handling discretionary access control for content and
metadata managed by WebDAV servers. WebDAV defines an Access Control
List (ACL) per resource. An ACL contains a set of Access Control
Entries (ACEs), where each ACE specifies a principal (i.e. user or
group of users) and a set of privileges that are granted to that
principal. When a principal tries to perform an operation on that
resource, the server evaluates the ACEs in the ACL to determine if
the principal has permission for that operation.
WebDAV also requires that an authentication mechanism be available
for the server to validate the identity of a principal. As a
minimum, all WebDAV compliant implementations are required to support
HTTP Digest Authentication.
A.2.1.2. Communication Resource Controls Primitives
Communications resources include bandwidth (upload/download) and
number of simultaneous connected clients (connections). WebDAV
supports communication resource control as described in
Appendix A.1.1.2.
A.2.1.3. Storage Resource Control Primitives
Storage resources include amount of memory and lifetime of storage.
WebDAV supports the concept of properties (which are metadata for a
resource). A property is either "live" or "dead". Live properties
include cases where a) the value of a property is protected and
maintained by the server, and b) the value of the property is
maintained by the client, but the server performs syntax checking on
submitted values. A dead property has its syntax and semantics
enforced by the client; the server merely records the value of the
property.
WebDAV supports a list of standardized properties [RFC4918] that are
useful for storage resource control. These include the self-
explanatory "creationdate", and "getcontentlength" properties. There
is also an operation called PROPFIND to retrieve all the properties
defined for the requested URI.
WebDAV also has a Quota and Size Properties mechanism defined in
[RFC4331] that can be used for storage control. Specifically, two
key properties are defined per resource: "quota-available-bytes" and
"quota-used-bytes".
WebDAV does not define protocol operation for storage resource
control. However servers typically perform this function via
implementation algorithms in conjunction with the storage related
properties discussed above.
A.2.2. WebDAV Support for DECADE Standard Transport Protocol Primitives
SDT is used to write objects and read (download) objects from a
DECADE server. The object can be either a self-contained object such
as a multimedia file or a chunk from a P2P system.
A.2.2.1. Writing Primitives
Writing involves uploading objects to the server. WebDAV supports
PUT and POST as described in Appendix A.1.2.1. WebDAV LOCK/UNLOCK
functionality is not needed as DECADE assumes immutable data objects.
Therefore, resources cannot be edited and so do not need to be
locked. This approach should help to greatly simplify DECADE
implementations as the LOCK/UNLOCK functionality is quite complex.
A.2.2.2. Downloading Primitives
Downloading involves fetching of an object from the server. WebDAV
supports GET and HEAD as described in Appendix A.1.2.2. WebDAV LOCK/
UNLOCK functionality is not needed as DECADE assumes immutable data
objects.
A.2.3. Other Operations
WebDAV supports DELETE as described in Appendix A.1.4. In addition
WebDAV supports COPY and MOVE methods. The COPY operation creates a
duplicate of the source resource identified by the Request-URI, in
the destination resource identified by the URI in the Destination
header.
The MOVE operation on a resource is the logical equivalent of a COPY,
followed by consistency maintenance processing, followed by a delete
of the source, where all three actions are performed in a single
operation. The consistency maintenance step allows the server to
perform updates caused by the move, such as updating all URLs, other
than the Request-URI that identifiesthe source resource, to point to
the new destination resource.
WebDAV also supports the concept of "collections" of resources to
support joint operations on related objects (e.g. file system
directories) within a server's namespace. For example, GET and HEAD
may be done on a single resource (as in HTTP) or on a collection.
The MKCOL operation is used to create a new collection. DECADE may
find the concept of collections to be useful if there is a need to
support directory like structures in DECADE.
WebDAV servers can be interfaced from an HTML-based user interface in
a web browser. However, it is frequently desirable to be able to
switch from an HTML-based view to a persentation provided by a native
WebDAV client, directly supporting WebDAV features. The method to
perform this in a platform-neutral mechanism is specified in the
WebDAV protocol for "mounting WebDAV servers" [RFC4709]. This type
of feature may also be attractive for DECADE clients.
A.2.4. Conclusions
WebDAV has a rich array of features that can provide a good base for
DRP and SDT for DECADE. An initial analysis finds that the following
WebDAV features will be useful for DECADE:
- access control
- properties (and PROPFIND operation)
- COPY/MOVE operations
- collections
- mounting WebDAV servers
It is recommended that the following WebDAV features NOT be used for
DECADE:
- LOCK/UNLOCK
Finally, some extensions to WebDAV may still be required to meet all
DECADE requirements. For example, defining a new WebDAV "time-to-
live" property may be useful for DECADE. Further analysis is
required to fully define the potential extensions to WebDAV to meet
all DECADE requirements.
Authors' Addresses Authors' Addresses
Richard Alimi Richard Alimi
Google Google
Email: ralimi@google.com Email: ralimi@google.com
Y. Richard Yang Y. Richard Yang
Yale University Yale University
 End of changes. 60 change blocks. 
166 lines changed or deleted 745 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/