draft-ietf-decade-arch-04.txt   draft-ietf-decade-arch-05.txt 
DECADE R. Alimi DECADE R. Alimi
Internet-Draft Google Internet-Draft Google
Intended status: Informational Y. Yang Intended status: Informational A. Rahman
Expires: May 3, 2012 Yale University Expires: September 13, 2012 InterDigital Communications, LLC
A. Rahman
InterDigital Communications, LLC
D. Kutscher D. Kutscher
NEC NEC
H. Liu Y. Yang
Yale University Yale University
October 31, 2011 March 12, 2012
DECADE Architecture DECADE Architecture
draft-ietf-decade-arch-04 draft-ietf-decade-arch-05
Abstract Abstract
Content Distribution Applications (e.g., P2P applications) are widely Content Distribution Applications (e.g., P2P applications) are widely
used on the Internet and make up a large portion of the traffic in used on the Internet and make up a large portion of the traffic in
many networks. One technique to improve the network efficiency of many networks. One technique to improve the network efficiency of
these applications is to introduce storage capabilities within the these applications is to introduce storage capabilities within the
networks; this is the capability to be provided by DECADE (DECoupled networks; this is the capability to be provided by a DECADE
Application Data Enroute). This document presents an architecture (DECoupled Application Data Enroute) compliant system. This document
for DECADE, discusses the underlying principles, and identifies core presents an architecture for DECADE, discusses the underlying
components and protocols for introducing in-network storage for these principles, and identifies key functionalities required for
applications. introducing in-network storage for these applications. In addition,
some examples are given to illustrate these concepts in an
informative manner.
Requirements Language Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
Status of this Memo Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79. provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF). Note that other groups may also distribute
other groups may also distribute working documents as Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at This Internet-Draft will expire on September 13, 2012.
http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html.
This Internet-Draft will expire on May 3, 2012.
Copyright Notice Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the Copyright (c) 2012 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5
2. Functional Entities . . . . . . . . . . . . . . . . . . . . . 6 2. Functional Entities . . . . . . . . . . . . . . . . . . . . . 5
2.1. DECADE Server . . . . . . . . . . . . . . . . . . . . . . 6 2.1. DECADE Server . . . . . . . . . . . . . . . . . . . . . . 6
2.2. DECADE Client . . . . . . . . . . . . . . . . . . . . . . 6 2.2. DECADE Client . . . . . . . . . . . . . . . . . . . . . . 6
2.3. DECADE Storage Provider . . . . . . . . . . . . . . . . . 6 2.3. DECADE Storage Provider . . . . . . . . . . . . . . . . . 6
2.4. Content Provider Using DECADE . . . . . . . . . . . . . . 6 2.4. Content Provider Using DECADE . . . . . . . . . . . . . . 6
2.5. Content Consumer Using DECADE . . . . . . . . . . . . . . 7 2.5. Content Consumer Using DECADE . . . . . . . . . . . . . . 6
2.6. Content Distribution Application . . . . . . . . . . . . . 7 2.6. Content Distribution Application . . . . . . . . . . . . . 7
3. Protocol Flow . . . . . . . . . . . . . . . . . . . . . . . . 7 3. Protocol Flow . . . . . . . . . . . . . . . . . . . . . . . . 7
3.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.2. An Example . . . . . . . . . . . . . . . . . . . . . . . . 9 3.2. An Example . . . . . . . . . . . . . . . . . . . . . . . . 9
4. Architectural Principles . . . . . . . . . . . . . . . . . . . 9 4. Architectural Principles . . . . . . . . . . . . . . . . . . . 9
4.1. Decoupled Control/Metadata and Data Planes . . . . . . . . 10 4.1. Decoupled Control/Metadata and Data Planes . . . . . . . . 10
4.2. Immutable Data Objects . . . . . . . . . . . . . . . . . . 11 4.2. Immutable Data Objects . . . . . . . . . . . . . . . . . . 10
4.3. Data Object Identifiers . . . . . . . . . . . . . . . . . 12 4.3. Data Object Identifiers . . . . . . . . . . . . . . . . . 11
4.4. Explicit Control . . . . . . . . . . . . . . . . . . . . . 12 4.4. DECADE Data Object Naming Scheme . . . . . . . . . . . . . 12
4.5. Resource and Data Access Control through User 4.5. Explicit Control . . . . . . . . . . . . . . . . . . . . . 13
Delegation . . . . . . . . . . . . . . . . . . . . . . . . 12 4.6. Resource and Data Access Control through User
5. System Components . . . . . . . . . . . . . . . . . . . . . . 13 Delegation . . . . . . . . . . . . . . . . . . . . . . . . 14
5.1. Content Distribution Application . . . . . . . . . . . . . 14 5. System Components . . . . . . . . . . . . . . . . . . . . . . 15
5.2. DECADE Server . . . . . . . . . . . . . . . . . . . . . . 16 5.1. Content Distribution Application . . . . . . . . . . . . . 15
5.3. Data Sequencing and Naming . . . . . . . . . . . . . . . . 18 5.2. DECADE Server . . . . . . . . . . . . . . . . . . . . . . 17
5.4. Token-based Authentication and Resource Control . . . . . 21 5.3. Data Sequencing and Naming . . . . . . . . . . . . . . . . 19
5.5. Discovery . . . . . . . . . . . . . . . . . . . . . . . . 22 5.4. Token-based Authentication and Resource Control . . . . . 22
6. DECADE Protocols . . . . . . . . . . . . . . . . . . . . . . . 23 5.5. Discovery . . . . . . . . . . . . . . . . . . . . . . . . 23
6.1. DECADE Resource Protocol (DRP) . . . . . . . . . . . . . . 23 6. DECADE Protocols . . . . . . . . . . . . . . . . . . . . . . . 24
6.2. Standard Data Transport (SDT) . . . . . . . . . . . . . . 26 6.1. DECADE Resource Protocol (DRP) . . . . . . . . . . . . . . 24
7. Server-to-Server Protocols . . . . . . . . . . . . . . . . . . 29 6.2. Standard Data Transfer (SDT) . . . . . . . . . . . . . . . 27
7.1. Operational Overview . . . . . . . . . . . . . . . . . . . 29 6.3. Server-to-Server Protocols . . . . . . . . . . . . . . . . 29
8. Potential Optimizations . . . . . . . . . . . . . . . . . . . 30 7. Security Considerations . . . . . . . . . . . . . . . . . . . 31
8.1. Pipelining to Avoid Store-and-Forward Delays . . . . . . . 30 7.1. Threat: System Denial of Service Attacks . . . . . . . . . 31
8.2. Deduplication . . . . . . . . . . . . . . . . . . . . . . 31 7.2. DECADE Protocol Security Threats . . . . . . . . . . . . . 32
9. Security Considerations . . . . . . . . . . . . . . . . . . . 32 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 33
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 33 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 33
11. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 33 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 34
12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 33 10.1. Normative References . . . . . . . . . . . . . . . . . . . 34
12.1. Normative References . . . . . . . . . . . . . . . . . . . 33 10.2. Informative References . . . . . . . . . . . . . . . . . . 34
12.2. Informative References . . . . . . . . . . . . . . . . . . 33 Appendix A. In-Network Storage Components Mapped to DECADE
Appendix A. Appendix: Evaluation of Some Candidate Existing Architecture . . . . . . . . . . . . . . . . . . . . 35
Protocols for DECADE DRP and SDT . . . . . . . . . . 34 A.1. Data Access Interface . . . . . . . . . . . . . . . . . . 35
A.1. HTTP . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 A.2. Data Management Operations . . . . . . . . . . . . . . . . 35
A.2. WEBDAV . . . . . . . . . . . . . . . . . . . . . . . . . . 37 A.3. Data Search Capability . . . . . . . . . . . . . . . . . . 35
A.3. CDMI . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 A.4. Access Control Authorization . . . . . . . . . . . . . . . 36
Appendix B. In-Network Storage Components Mapped to DECADE A.5. Resource Control Interface . . . . . . . . . . . . . . . . 36
Architecture . . . . . . . . . . . . . . . . . . . . 42 A.6. Discovery Mechanism . . . . . . . . . . . . . . . . . . . 36
B.1. Data Access Interface . . . . . . . . . . . . . . . . . . 43 A.7. Storage Mode . . . . . . . . . . . . . . . . . . . . . . . 36
B.2. Data Management Operations . . . . . . . . . . . . . . . . 43
B.3. Data Search Capability . . . . . . . . . . . . . . . . . . 43 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 36
B.4. Access Control Authorization . . . . . . . . . . . . . . . 43
B.5. Resource Control Interface . . . . . . . . . . . . . . . . 43
B.6. Discovery Mechanism . . . . . . . . . . . . . . . . . . . 43
B.7. Storage Mode . . . . . . . . . . . . . . . . . . . . . . . 44
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 44
1. Introduction 1. Introduction
Content Distribution Applications are widely used on the Internet Content Distribution Applications (e.g., P2P applications) are widely
today to distribute data, and they contribute a large portion of the used on the Internet today to distribute data, and they contribute a
traffic in many networks. The DECADE architecture described in this large portion of the traffic in many networks. The DECADE-compatible
document enables such applications to leverage in-network storage to architecture described in this document enables such applications to
achieve more efficient content distribution. Specifically, in many leverage in-network storage to achieve more efficient content
subscriber networks, it can be expensive to upgrade network equipment distribution. Specifically, in many subscriber networks, it can be
in the "last-mile", because it can involve replacing equipment and expensive to upgrade network equipment in the "last-mile", because it
upgrading wiring at individual homes, businesses, and devices such as can involve replacing equipment and upgrading wiring at individual
DSLAMs (Digital Subscriber Line Access Multiplexers) and CMTSs (Cable homes, businesses, and devices such as DSLAMs (Digital Subscriber
Modem Termination Systems) in remote locations. Therefore, it can be Line Access Multiplexers) and CMTSs (Cable Modem Termination Systems)
cheaper to upgrade the core infrastructure, which involves fewer in remote locations. Therefore, it may be cheaper to upgrade the
components that are shared by many subscribers. See core infrastructure, which involves fewer components that are shared
[I-D.ietf-decade-problem-statement] for a more complete discussion of by many subscribers. See [I-D.ietf-decade-problem-statement] for a
the problem domain and general discussions of the capabilities to be more complete discussion of the problem domain and general
provided by DECADE. discussions of the capabilities to be provided by a DECADE-compatible
system.
This document presents an architecture for providing in-network This document presents an architecture for providing in-network
storage that can be integrated into Content Distribution storage that can be integrated into Content Distribution
Applications. The primary focus is P2P-based content distribution, Applications. The primary focus is P2P-based content distribution,
but the architecture may be useful to other applications with similar but the architecture may be useful to other applications with similar
characteristics and requirements. See [I-D.ietf-decade-reqs] for a characteristics and requirements. See [I-D.ietf-decade-reqs] for a
definition of the target applications supported by DECADE. definition of the target applications supported by a DECADE-
compatible system.
The design philosophy of the DECADE architecture is to provide only
the core functionalities that are needed for applications to make use
of in-network storage. Focusing on only core functionalities, DECADE
may be simpler and easier to support by service providers. If more
complex functionalities are needed by a certain application or class
of applications, it may be layered on top of DECADE.
DECADE will leverage existing data transport and application-layer
protocols. The design is to work with a small set of alternative
IETF protocols. In this document, we use "data transport" to refer
to a protocol that is used to read data from and write data into
DECADE in-network storage.
This document proceeds in two steps. First, it details the core The approach of this document is to define the core functionalities
architectural principles that we use to guide the DECADE design. and protocol requirements that are needed to support in-network
Next, given these core principles, this document presents the core storage in a DECADE-compatible system. The protocol themselves are
components of the DECADE architecture and identifies the usage of not selected or designed. Also, if more complex functionalities are
existing protocols and where there is a need for new protocol needed, they should be layered on top of the DECADE-compatible system
development. in an application specific manner.
This document does not define any new protocol. Instead, when Some illustrative examples are given to help the reader understand
identifying the need for a new protocol, it presents an abstract certain concepts. These examples are purely informational and are
design including the necessary functions of that protocol (including not meant to guide or constrain future protocol design or selection.
rationale) in order to guide future protocol development.
2. Functional Entities 2. Functional Entities
This section defines the functional entities involved in a DECADE This section defines the functional entities involved in a DECADE
system. Functional entities can be classified as follows: system. Functional entities are classified as follows:
o A physical or logical component in the DECADE architecture: DECADE o A physical or logical component in the DECADE architecture: DECADE
Client, DECADE Server, Content Distribution Application and Client, DECADE Server, Content Distribution Application and
Application End Point; Application End Point;
o Operator of a physical or logical component in the DECADE o Operator of a physical or logical component in the DECADE
architecture: DECADE Storage Provider; and architecture: DECADE Storage Provider; and
o Source or sink of content distributed via the DECADE architecture: o Source or sink of content distributed via the DECADE architecture:
DECADE Content Provider, and DECADE Content Consumer. DECADE Content Provider, and DECADE Content Consumer.
skipping to change at page 6, line 49 skipping to change at page 6, line 39
A DECADE storage provider, possibly in cooperation with one or more A DECADE storage provider, possibly in cooperation with one or more
network providers, determines deployment locations for DECADE servers network providers, determines deployment locations for DECADE servers
and determines the available resources for each. and determines the available resources for each.
2.4. Content Provider Using DECADE 2.4. Content Provider Using DECADE
A content provider using DECADE accesses DECADE storage servers (by A content provider using DECADE accesses DECADE storage servers (by
way of a DECADE client) to upload and manage data. Such a content way of a DECADE client) to upload and manage data. Such a content
provider can access one or more storage servers. Such a content provider can access one or more storage servers. Such a content
provider may be a single process or a distributed application (e.g., provider may be a single process or a distributed application (e.g.,
in a P2P scenario), and may either be fixed or mobile. in a P2P scenario), and may be either fixed or mobile.
2.5. Content Consumer Using DECADE 2.5. Content Consumer Using DECADE
A content consumer using DECADE accesses DECADE storage servers (by A content consumer using DECADE accesses DECADE storage servers (by
way of a DECADE client). A content consumer can access one or more way of a DECADE client). A content consumer can access one or more
DECADE storage servers. A content consumer may be a single process DECADE storage servers. A content consumer may be a single process
or a distributed application (e.g., in a P2P scenario), and may or a distributed application (e.g., in a P2P scenario), and may be
either be fixed or mobile. An instance of a distributed application, either fixed or mobile. An instance of a distributed application,
such as a P2P application, may both provide content to and consume such as a P2P application, may both provide content to and consume
content from DECADE storage servers. content from DECADE storage servers.
2.6. Content Distribution Application 2.6. Content Distribution Application
A content distribution application (as a target application for A content distribution application (as a target application for
DECADE as described in [I-D.ietf-decade-reqs]) is a distributed DECADE as described in [I-D.ietf-decade-reqs]) is a distributed
application designed for dissemination of a possibly-large data set application designed for dissemination of a possibly-large data set
to multiple consumers. Content Distribution Applications typically to multiple consumers. Content Distribution Applications typically
divide content into smaller blocks for dissemination. divide content into smaller blocks for dissemination.
The term Application Developer refers to the developer of a
particular Content Distribution Application.
2.6.1. Application End-Point 2.6.1. Application End-Point
An Application End-Point is an instance of a Content Distribution An Application End-Point is an instance of a Content Distribution
Application that makes use of DECADE server(s). A particular Application that makes use of DECADE server(s). A particular
Application End-Point may be a DECADE Content Provider, a DECADE Application End-Point may be a DECADE Content Provider, a DECADE
Content Consumer, or both. For example, an Application End-Point may Content Consumer, or both. For example, an Application End-Point may
be an instance of a video streaming client, or it may be the source be an instance of a video streaming client, or it may be the source
providing the video to a set of clients. providing the video to a set of clients.
An Application End-Point need not be actively transferring data with An Application End-Point need not be actively transferring data with
other Application End-Points to interact with the DECADE storage other Application End-Points to interact with the DECADE storage
system. That is, an End-Point may interact with the DECADE storage system. That is, an End-Point may interact with the DECADE storage
servers as an offline activity. servers as an offline activity.
3. Protocol Flow 3. Protocol Flow
3.1. Overview 3.1. Overview
The DECADE Architecture uses two protocols, as shown in Figure 1. A DECADE-compatible system supports two logical protocols, as shown
First, the DECADE Resource Protocol is responsible for communication in Figure 1. First, the DECADE Resource Protocol (DRP) is
of access control and resource scheduling policies from DECADE Client responsible for communication of access control and resource
to DECADE Server, as well as between DECADE Servers. The DECADE scheduling policies from a DECADE Client to a DECADE Server, as well
Architecture includes exactly one DRP for interoperability and a as between DECADE Servers. A DECADE-compatible system includes
common format through which these policies can be communicated. exactly one DRP for interoperability and a common format through
which these policies can be communicated.
Native Application Native Application
.-------------. Protocol(s) .-------------. .-------------. Protocol(s) .-------------.
| Application | <------------------> | Application | | Application | <------------------> | Application |
| End-Point | | End-Point | | End-Point | | End-Point |
| | | | | | | |
| .--------. | | .--------. | | .--------. | | .--------. |
| | DECADE | | | | DECADE | | | | DECADE | | | | DECADE | |
| | Client | | | | Client | | | | Client | | | | Client | |
| `--------' | | `--------' | | `--------' | | `--------' |
`-------------' `-------------' `-------------' `-------------'
| ^ | ^ | ^ | ^
DECADE | | Standard | | DECADE | | Standard | |
Resource | | Data DRP | | SDT Resource | | Data DRP | | SDT
Protocol | | Transport | | Protocol | | Transfer | |
(DRP) | | (SDT) | | (DRP) | | (SDT) | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
v V v V v V v V
.=============. DRP .=============. .=============. DRP .=============.
| DECADE | <------------------> | DECADE | | DECADE | <------------------> | DECADE |
| Server | <------------------> | Server | | Server | <------------------> | Server |
`=============' SDT `=============' `=============' SDT `============='
Figure 1: Generic Protocol Flow Figure 1: Generic Protocol Flow
Second, Standard Data Transport protocols (e.g., HTTP, WebDAV, or Second, a Standard Data Transport protocol (SDT) is used to transfer
CDMI) are used to transfer data objects to and from a DECADE Server. data objects to and from a DECADE Server. A DECADE-compatible system
The DECADE architecture may be used with multiple standard data may support multiple SDT's. If there are multiple SDT's, a
transports. negotiation mechanism is used to determine a suitable data transfer
protocol between a DECADE Client and DECADE Server.
Decoupling the protocols in this way allows DECADE to directly
utilize existing standard data transports, as well as allowing both
DECADE and DRP to evolve independently from data transports.
It is also important to note that the two protocols do not need to be The two protocols (DRP and SDT) may or may not be seperate on the
separate on the wire. For example, DRP messages may be piggybacked wire. For example, DRP messages may be piggy-backed within some
within some extension fields provided by certain data transport extension fields provided by certain data transfer protocols. In
protocols. In such a scenario, DRP is technically a data structure such a scenario, DRP is technically a data structure (transported by
(transported by other protocols), but it can still be considered as a other protocols), but it can still be considered as a logical
logical protocol that provides the services of configuring DECADE protocol that provides the services of configuring DECADE resource
resource usage. Hence, this document considers SDT and DRP as two usage. Hence, this document considers SDT and DRP as two separate,
separate, logical functional components for clarity. logical functional components for clarity. The abstract properties
of the SDT and DRP are oultined but the final selection of these
protocols is left to future steps.
3.2. An Example 3.2. An Example
Before discussing details of the architecture, this section provides This section provides an example of steps in a data transfer scenario
an example data transfer scenario to illustrate how the DECADE involving an in-network storage system. We assume that Application
Architecture can be applied. End-Point B (the receiver) is requesting a data object from
Application End-Point A (the sender). Let S(A) denote A's DECADE-
In this example, we assume that Application End-Point B (the compatible storage server. There are multiple usage scenarios (by
receiver) is requesting a data object from Application End-Point A choice of the Content Distribution Application). For simplicity of
(the sender). Let S(A) denote A's DECADE storage server. There are introduction, we design this example to use only a single DECADE-
multiple usage scenarios (by choice of the Content Distribution compatible Server; Section 6.3 details a case when both A and B wish
Application). For simplicity of introduction, we design the example to employ DECADE-compatible Servers.
to use only a single DECADE Server; Section 7 details a case when
both A and B wish to employ DECADE Servers.
When an Application End-Point wishes to use its DECADE storage When an Application End-Point wishes to use its DECADE-compatible
server, it provides a token (see Section 6.1.2 for details) to the storage server, it provides a token to the other Application End-
other Application End-Point. The token is sent using the Content Point. The token is sent using the Content Distribution
Distribution Application's native protocol. Application's native protocol.
The steps of the example are illustrated in Figure 2. First, B The steps of the example are illustrated in Figure 2. First, B
requests a data object from A using their native protocol. Next, A requests a data object from A using their native protocol. Next, A
uses the DECADE Resource Protocol (DRP) to obtain a token from its uses the DECADE-compatible Resource Protocol (DRP) to obtain a token.
DECADE storage server, S(A). A then provides the token to B (again, There are multiple ways for A to obtain the token: compute locally,
or request from its DECADE-compatible storage server, S(A). See
Section 6.1.2 for details. A then provides the token to B (again,
using their native protocol). Finally, B provides the token to S(A) using their native protocol). Finally, B provides the token to S(A)
via DRP, and requests and downloads the data object via a Standard via DRP, and requests and downloads the data object via a Standard
Data Transport (SDT). Data Transport (SDT).
.----------. .----------.
----------> | S(A) | <------ 2. Obtain --------> | S(A) | <------
2. Obtain / `----------' \ 4. Request and Token / `----------' \ 4. Request and
Token / \ Download Object (DRP) / \ Download Object
(DRP) / \ (DRP + SDT) Locally / \ (DRP + SDT)
v 1. App request v or From / \
.-------------. <--------------------------- .-------------. S(A) v 1. App Request v
| End-Point A | | End-Point B | .-------------. <--------------------------- .-------------.
`-------------' ---------------------------> `-------------' | End-Point A | | End-Point B |
3. App response (token) `-------------' ---------------------------> `-------------'
3. App Response (token)
Figure 2: Download from Storage Server Figure 2: Download from Storage Server
4. Architectural Principles 4. Architectural Principles
We identify the following key principles. We identify the following key principles.
4.1. Decoupled Control/Metadata and Data Planes 4.1. Decoupled Control/Metadata and Data Planes
The DECADE infrastructure is intended to support multiple content A DECADE-compatible system will support multiple content distribution
distribution applications. A complete content distribution applications. A complete content distribution application implements
application implements a set of control and management functions a set of "control plane" functions including content search, indexing
including content search, indexing and collection, access control, ad and collection, access control, ad insertion, replication, request
insertion, replication, request routing, and QoS scheduling. An routing, and QoS scheduling. Different content distribution
observation of DECADE is that different content distribution applications will have unique considerations designing the control
applications can have unique considerations designing the control and plane functions:
signaling functions:
o Metadata Management Scheme: Traditional file systems provide a o Metadata Management Scheme: Traditional file systems provide a
standard metadata abstraction: a recursive structure of standard metadata abstraction: a recursive structure of
directories to offer namespace management; each file is an opaque directories to offer namespace management; each file is an opaque
byte stream. In content distribution, applications may use byte stream. In content distribution, applications may use
different metadata management schemes. For example, one different metadata management schemes. For example, one
application may use a sequence of blocks (e.g., for file sharing), application may use a sequence of blocks (e.g., for file sharing),
while another application may use a sequence of frames (with while another application may use a sequence of frames (with
different sizes) indexed by time. different sizes) indexed by time.
o Resource Scheduling Algorithms: a major competitive advantage of o Resource Scheduling Algorithms: a major competitive advantage of
many successful P2P systems is their substantial expertise in many successful P2P systems is their substantial expertise in
achieving highly efficient utilization of peer and infrastructural achieving highly efficient utilization of peer and infrastructural
resources. For instance, many live P2P systems have their resources. For instance, many streaming P2P systems have their
specific algorithms in constructing topologies to achieve low- specific algorithms in constructing topologies to achieve low-
latency, high-bandwidth streaming. They continue to fine-tune latency, high-bandwidth streaming. They continue to fine-tune
such algorithms. such algorithms.
Given the diversity of control-plane functions, in-network storage Given the diversity of control plane functions, in-network storage
should export basic mechanisms and allow as much flexibility as should export basic mechanisms and allow as much flexibility as
possible to the control planes to implement specific policies. This possible to the control plane to implement specific policies. This
conforms to the end-to-end systems principle and allows innovation conforms to the end-to-end systems principle and allows innovation
and satisfaction of specific business goals. and satisfaction of specific business goals.
Decoupling control plane and data plane is not new. For example, Decoupling control plane and data plane is not new. For example,
OpenFlow is an implementation of this principle for Internet routing, OpenFlow [OpenFlow] is an implementation of this principle for
where the computation of the forwarding table and the application of Internet routing, where the computation of the forwarding table and
the forwarding table are separated. Google File System the application of the forwarding table are separated. Google File
[GoogleFileSystem] applies the principle to file system design, by System [GoogleFileSystem] applies the principle to file system
utilizing the Master to handle the meta-data management, and the design, by utilizing the Master to handle the meta-data management,
Chunk Servers to handle the data plane functions (i.e., read and and the Chunk Servers to handle the data plane functions (i.e., read
write of chunks of data). NFSv4.1's pNFS extension [RFC5661] also and write of chunks of data). NFSv4.1's pNFS extension [RFC5661]
implements this principle. also implements this principle.
Note that applications may have different Data Plane implementations
in order to support particular requirements (e.g., low latency). In
order to provide interoperability, the DECADE architecture does not
intend to enable arbitrary data transport protocols. However, the
architecture may allow for more-than-one data transport protocols to
be used.
Also note that although an application's existing control plane
functions remain implemented within the application, the particular
implementation may need to be adjusted to support DECADE.
4.2. Immutable Data Objects 4.2. Immutable Data Objects
A property of bulk contents to be broadly distributed is that they A property of bulk contents to be broadly distributed is that they
typically are immutable -- once a piece of content is generated, it typically are immutable -- once content is generated, it is typically
is typically not modified. It is not common that bulk contents such not modified. It is not common that bulk contents such as video
as video frames and images need to be modified after distribution. frames and images need to be modified after distribution.
Many content distribution applications divide content objects into
blocks for two reasons: (1) multipath: different blocks may be
fetched from different content sources in parallel, and (2) faster
recovery and verification: individual blocks may be recovered and
verified. Typically, applications use a block size larger than a
single packet in order to reduce control overhead.
Common applications using the aforementioned data model include P2P
streaming (live and video-on-demand) and P2P file-sharing. However,
other additional types of applications may match this model.
DECADE adopts a design in which immutable data objects may be stored
at a storage server. Applications may consider existing blocks as
DECADE data objects, or they may adjust block sizes before storing in
a DECADE server.
Focusing on immutable data blocks in the data plane can substantially Focusing on immutable data in the data plane can substantially
simplify the data plane design, since consistency requirements can be simplify the data plane design, since consistency requirements can be
relaxed. It also allows effective reuse of data blocks and de- relaxed. It also allows effective reuse of data and de-duplication
duplication of redundant data. of redundant content.
Depending on its specific requirements, an application may store data Depending on its specific requirements, an application may store
in DECADE servers such that each data object is completely self- immutable data objects in DECADE-compatible servers such that each
contained (e.g., a complete, independently decodable video segment). data object is completely self-contained (e.g., a complete,
An application may also divide data into chunks that require independently decodable video segment). An application may also
application level assembly. The DECADE architecture and protocols divide data into blocks that require application level assembly.
are agnostic to the nature of the data objects and do not specify a Many content distribution applications divide bulk content into
fixed size for them. blocks for two reasons: (1) multipath: different blocks may be
fetched from different sources in parallel, and (2) faster recovery
and verification: individual blocks may be recovered and verified.
Typically, applications use a block size larger than a single packet
in order to reduce control overhead.
Note that immutable content may still be deleted. Also note that A DECADE-compatible system is agnostic to the nature of the data
immutable data blocks do not imply that contents cannot be modified. objects and does not specify a fixed size for them, though a protocol
For example, a meta-data management function of the control plane may specification based on this architecture may prescribe requirements
associate a name with a sequence of immutable blocks. If one of the on minimum and maximum sizes by compliant implementatations.
blocks is modified, the meta-data management function changes the Applications may consider existing blocks as data objects, or they
mapping of the name to a new sequence of immutable blocks. may adjust block sizes before storing in the DECADE-compatible
server.
Throughout this document, all the data objects/blocks are referred as Immutable content may still be deleted. Applications may support
modification of existing data stored at a DECADE-compatible server
through a combination of storing new data objects and deleting
existing data objects. For example, a meta-data management function
of the control plane may associate a name with a sequence of
immutable blocks. If one of the blocks is modified, the meta-data
management function changes the mapping of the name to a new sequence
of immutable blocks.
Throughout this document, all data objects/blocks are referred to as
immutable data objects/blocks. immutable data objects/blocks.
4.3. Data Object Identifiers 4.3. Data Object Identifiers
Objects that are stored in a DECADE storage server can be accessed by Objects that are stored in a DECADE-compatible storage server can be
DECADE content consumers by a resource identifier that has been accessed by content consumers via a data object identifier that has
assigned within a certain application context. been assigned within a certain application context and that is unique
within that application context.
Because a DECADE content consumer can access more than one storage A content consumer may be able to access more than one storage server
server within a single application context, a data object that is within a single application context. A data object that is
replicated across different storage servers managed by a DECADE replicated across different storage servers managed by a DECADE-
storage provider, can be accessed by a single identifier. compatible storage provider can still be accessed by a single
identifier.
Note that since data objects are immutable, it is possible to support Since data objects are immutable, it is possible to support
persistent identifiers for data objects. persistent identifiers for data objects.
4.4. Explicit Control Data object identifiers for data objects MUST be created by content
providers that upload the objects to DECADE servers.
For some applications it is important that DECADE clients and servers
are able to validate the name-object binding for a data object, i.e.,
by verifying that a received object really corresponds to the name
that was used for requesting it (or that was provided by a sender).
Data object identifiers can support name-content binding validation
by providing message digests or so-called self-certifying naming
information -- if a specific application has this requirement.
4.4. DECADE Data Object Naming Scheme
The DECADE architecture is based on the data object identifier
properties as described in Section 4.3, but does neither specify a
specific scheme nor specific name-object binding validation functions
(for example, hash functions). Different applications will have
specific requirements regarding security (for example, cryptographic
strength of hash functions), performance (for example, larger static
objects vs streaming) etc.
The details of such naming schemes will be provided by DECADE
protocol/naming specifications. This documents describes the scheme
on a semantic level will but specific SDTs and DRPs may use specific
representations. The naming scheme meets the following general
requirements:
o Different name-object binding validation mechanisms are supported;
o an application (DECADE content provider) can decide what mechanism
to use (or to not provide name-object validation -- for example if
authenticity and integrity can be ascertained by alternative
means);
o simple (digest hash based) name-object binding validation is
supported; and
o applications are able to construct unique names (with high
probability) without requiring a registry or other forms of
coordination; and
o names are self-describing so that a receiving entity (DECADE
content consumer) knows what hash function (for example) to use
for validating name-object binding.
For most content distribution scenarios, it will be appropriate to
derive the name of a data object from the hash over the data object's
content (the raw bytes), which is made possible by the fact that
DECADE-compatible objects are immutable. But there maybe other
applications such as live streaming where object/chunk names are not
based on hashes but rather on a enumeration scheme. The DECADE
naming scheme also enables those applications to construct unique
names.
In order to enable the uniqueness, flexibility and self-describing
properties, the DECADE naming scheme provides the following name
elements:
o a "type" field that indicates the name-object validation function
type (for example, "sha-256");
o cryptographic data (such as an object hash) that corresponds to
the type information; and
o (optionally) additional information such as application context or
publisher information.
The specific format of the name (e.g., encoding, hash algorithms,
etc) is out of scope of this document, and left for protocol
specification.
The DECADE naming scheme can be used in scenarios where a client
knows the name of a data object before it is completely stored at the
server. This allows for particular optimizations, such as
advertising data object while the data object is being stored,
removing store-and-forward delays. For example, a client A may
simultaneously begin storing an object to a server, and advertise
that the object is available to client B. If it is supported by the
server, client B may begin downloading the object before A is
finished storing the object.
If object names are not based on content hashes, DECADE names can
also be used in scenarios, where a client knows the name of a data
object before it is locally created.
4.5. Explicit Control
To support the functions of an application's control plane, To support the functions of an application's control plane,
applications must be able to know and control which data is stored at applications must be able to know and coordinate which data is stored
particular locations. Thus, in contrast with content caches, at particular servers. Thus, in contrast with content caches,
applications are given explicit control over the placement (selection applications are given explicit control over the placement (selection
of a DECADE server), deletion (or expiration policy), and access of a DECADE-compatible server), deletion (or expiration policy), and
control for stored data. access control for stored data.
Consider deletion/expiration policy as a simple example. An Consider deletion/expiration policy as a simple example. An
application may require that a DECADE server store content for a application may require that a server store content for a relatively
relatively short period of time (e.g., for live-streaming data). short period of time (e.g., for live-streaming data). Another
Another application may need to store content for a longer duration application may need to store content for a longer duration (e.g.,
(e.g., for video-on-demand). for video-on-demand).
4.5. Resource and Data Access Control through User Delegation 4.6. Resource and Data Access Control through User Delegation
DECADE provides a shared infrastructure to be used by multiple A DECADE-compatible system provides a shared infrastructure to be
tenants of multiple content distribution applications. Thus, it used by multiple tenants of multiple content distribution
needs to provide both resource and data access control. applications. Thus, it needs to provide both resource and data
access control.
4.5.1. Resource Allocation 4.6.1. Resource Allocation
There are two primary interacting entities in the DECADE There are two primary interacting entities in a DECADE-compatible
architecture. First, Storage Providers control where DECADE storage system. First, Storage Providers coordinate where storage servers
servers are provisioned and their total available resources. Second, are provisioned and their total available resources. Second,
Applications control data transfers amongst available DECADE servers Applications coordinate data transfers amongst available servers and
and between DECADE servers and end-points. A form of isolation is between servers and end-points. A form of isolation is required to
required to enable concurrently-running Applications to each enable concurrently-running Applications to each explicitly manage
explicitly manage its own content and share of resources at the its own content and share of resources at the available servers.
available servers.
The Storage Provider delegates the management of the resources at a The Storage Provider delegates the management of the resources on a
DECADE server to one or more applications. Applications are able to server to applications. It means applications are able to explicitly
explicitly and independently manage their own shares of resources. and independently manage their own shares of resources on a DECADE
server.
4.5.2. User Delegations 4.6.2. User Delegations
Storage providers have the ability to explicitly manage the entities Storage providers have the ability to explicitly manage the entities
allowed to utilize the resources at a DECADE server. This capability allowed to utilize the resources at a DECADE-compatible server. This
is needed for reasons such as capacity-planning and legal capability is needed for reasons such as capacity-planning and legal
considerations in certain deployment scenarios. considerations in certain deployment scenarios.
To provide a scalable way to manage applications granted resources at The server grants a share of the resources to a user. The user may
a DECADE server, we consider an architecture that adds a layer of in turn share the granted resources amongst multiple applications.
indirection. Instead of granting resources to an application, the The share of resources granted by a storage provider is called a User
DECADE server grants a share of the resources to a user. The user Delegation.
may in turn share the granted resources amongst multiple
applications. The share of resources granted by a storage provider
is called a User Delegation.
As a simple example, a DECADE Server operated by an ISP may be As a simple example, a DECADE-compatible Server operated by an ISP
configured to grant each ISP Subscriber 1.5 Mbps of bandwidth. The may be configured to grant each ISP Subscriber 1.5 Mbps of bandwidth.
ISP Subscriber may in turn divide this share of resources amongst a The ISP Subscriber may in turn divide this share of resources amongst
video streaming application and file-sharing application which are a video streaming application and file-sharing application which are
running concurrently. running concurrently.
In general, a User Delegation may be granted to an end-user (e.g., an In general, a User Delegation may be granted to an end-user (e.g., an
ISP subscriber), a Content Provider, or an Application Provider. A ISP subscriber), a Content Provider, or an application. A particular
particular instance of an application may make use of the storage instance of an application may make use of the storage resources:
resources:
o granted to the end-user (with the end-user's permission), o granted to the end-user (with the end-user's permission),
o granted to the Content Provider (with the Content Provider's o granted to the Content Provider (with the Content Provider's
permission), and/or permission), and/or
o granted to the Application Provider. o granted to the application.
5. System Components 5. System Components
The primary focus of this document is the architectural principals The primary focus of this document is the architectural principles
and the system components that implement them. While certain system and the system components that implement them. While certain system
components might differ amongst implementations, the document details components might differ amongst implementations, the document details
the major components and their overall roles in the architecture. the major components and their overall roles in the architecture.
To keep the scope narrow, we only discuss the primary components To keep the scope narrow, we only discuss the primary components
related to protocol development. Particular deployments may require related to protocol development. Particular deployments may require
additional components (e.g., monitoring and accounting at a DECADE additional components (e.g., monitoring and accounting at a server),
server), but they are intentionally omitted from this document. but they are intentionally omitted from this document.
5.1. Content Distribution Application 5.1. Content Distribution Application
Content Distribution Applications have many functional components. Content Distribution Applications have many functional components.
For example, many P2P applications have components and algorithms to For example, many P2P applications have components and algorithms to
manage overlay topology management, piece selection, etc. In manage overlay topology management, rate allocation, piece selection,
supporting DECADE, it may be advantageous for an application etc. In this document, we focus on the components directly employed
developer to consider DECADE in the implementation of these to support a DECADE-compatible system.
components. However, in this architecture document, we focus on the
components directly employed to support DECADE.
Figure 3 illustrates the components discussed in this section from Figure 3 illustrates the components discussed in this section from
the perspective of a single Application End-Point and their relation the perspective of a single Application End-Point.
to DECADE.
Native Protocol(s) Native Protocol(s)
(with other Application End-Points) (with other Application End-Points)
.---------------------> .--------------------->
| |
| |
.----------------------------------------------------------. .----------------------------------------------------------.
| Application End-Point | | Application End-Point |
| .------------. .-------------------. | | .------------. .-------------------. |
| | App-Layer | ... | App Data Assembly | | | | App-Layer | ... | App Data Assembly | |
skipping to change at page 15, line 34 skipping to change at page 16, line 34
| | | | Access | | Sharing | | | | Sched. | | Index | | | | | | | | Access | | Sharing | | | | Sched. | | Index | | | |
| | | | Policy | | Policy | | | | | | | | | | | | | | Policy | | Policy | | | | | | | | | |
| | | '--------' `----------' | | `--------' `-------' | | | | | | '--------' `----------' | | `--------' `-------' | | |
| | `-------------------------' `----------------------' | | | | `-------------------------' `----------------------' | |
| | | ^ | | | | | ^ | |
| `------------ | ----------------- | -------------------' | | `------------ | ----------------- | -------------------' |
`-------------- | ----------------- | ---------------------' `-------------- | ----------------- | ---------------------'
| | | |
| DECADE | Standard | DECADE | Standard
| Resource | Data | Resource | Data
| Protocol | Transport | Protocol | Transfer
| (DRP) | (SDT) | (DRP) | (SDT)
v V v V
Figure 3: Application Components Figure 3: Application Components
5.1.1. Data Assembly 5.1.1. Data Assembly
DECADE is primarily designed to support applications that can divide A DECADE-compatible system is geared towards supporting applications
distributed contents into data objects. To accomplish this, that can divide distributed contents into data objects. To
applications include a component responsible for creating the accomplish this, applications include a component responsible for
individual data objects before distribution and then re-assembling creating the individual data objects before distribution and then re-
data objects at the Content Consumer. We call this component assembling data objects at the Content Consumer. We call this
Application Data Assembly. The specific implementation is entirely component Application Data Assembly.
decided by the application.
In producing and assembling the data objects, two important In producing and assembling the data objects, two important
considerations are sequencing and naming. The DECADE architecture considerations are sequencing and naming. A DECADE-compatible system
assumes that applications implement this functionality themselves. assumes that applications implement this functionality themselves.
See Section 5.3 for further discussion. See Section 5.3 for further discussion.
5.1.2. Native Protocols 5.1.2. Native Protocols
Applications may still use existing protocols. In particular, an Applications may still use existing protocols. In particular, an
application may reuse existing protocols primarily for control/ application may reuse existing protocols primarily for control/
signaling. However, an application may still retain its existing signaling. However, an application may still retain its existing
data transport protocols, in addition to DECADE as the data transport data transfer protocols in addition to employing a data transfer
protocol. This can be important for applications that are designed protocol with DECADE-complliant support. This is important for
to be highly robust (e.g., if DECADE servers are unavailable). applications that are designed to be highly robust (e.g., if in-
network servers are unavailable).
5.1.3. DECADE Client 5.1.3. DECADE Client
An application may be modified to support DECADE. We call the layer An application needs to be modified to support a DECADE-compatible
providing the DECADE support to an application the DECADE Client. It system. The DECADE Client provides the local support to an
is important to note that a DECADE Client need not be embedded into application. A DECADE Client need not be embedded into the
an application. It could be implemented alone, or could be application. It could be implemented alone, or could be integrated
integrated in other entities such as network devices themselves. in other entities such as network devices themselves.
5.1.3.1. Resource Controller 5.1.3.1. Resource Controller
Applications may have different Resource Sharing Policies and Data Applications may have different Resource Sharing Policies and Data
Access Policies to control their resource and data in DECADE servers. Access Policies to control their resource and data in DECADE-
These policies can be existing policies of applications (e.g., tit- compatible servers. These policies can be existing policies of
for-tat) or custom policies adapted for DECADE. The specific applications (e.g., tit-for-tat) or custom policies. The specific
implementation is decided by the application. implementation is decided by the application.
5.1.3.2. Data Controller 5.1.3.2. Data Controller
DECADE is designed to decouple the control and the data transport of A DECADE-compatible system decouples the control and the data
applications. Data transport between applications and DECADE servers transfer of applications. A Data Scheduling component schedules data
uses standard data transport protocols. A Data Scheduling component transfers according to network conditions, available Servers, and/or
schedules data transfers according to network conditions, available available Server resources. The Data Index indicates data available
DECADE Servers, and/or available DECADE Server resources. The Data at remote servers. The Data Index (or a subset of it) may be
Index indicates data available at remote DECADE servers. The Data advertised to other Application End-Points. A common use case for
Index (or a subset of it) may be advertised to other Application End- this is to provide the ability to locate data amongst a distributed
Points. A common use case for this is to provide the ability to set of Application End-Points (i.e., a data search mechanism).
locate data amongst a distributed set of Application End-Points
(i.e., a data search mechanism).
5.2. DECADE Server 5.2. DECADE Server
A DECADE Server stores data from Application End-Points, and provides A DECADE-compatible Server stores data from Application End-Points,
control and access of those data to Application End-Points. Note and provides control and access of those data to Application End-
that a DECADE Server is not necessarily a single physical machine, it Points. A Server is not necessarily a single physical machine, it
could also be implemented as a cluster of machines. could also be implemented as a cluster of machines.
| | | |
| DECADE | Standard | DECADE | Standard
| Resource | Data | Resource | Data
| Protocol | Transport | Protocol | Transfer
| (DRP) | (SDT) | (DRP) | (SDT)
| | | |
.= | ================= | ======================. .= | ================= | ======================.
| | v | | | v |
| | .----------------. | | | .----------------. |
| |----> | Access Control | <--------. | | |----> | Access Control | <--------. |
| | `----------------' | | | | `----------------' | |
| | ^ | | | | ^ | |
| | | | | | | | | |
| | v | | | | v | |
skipping to change at page 17, line 36 skipping to change at page 18, line 36
| | Data Store | | Delegation | | | | Data Store | | Delegation | |
| `-----------------' | Management | | | `-----------------' | Management | |
| DECADE Server `------------' | | DECADE Server `------------' |
`==============================================' `=============================================='
Figure 4: DECADE Server Components Figure 4: DECADE Server Components
5.2.1. Access Control 5.2.1. Access Control
An Application End-Point can access its own data or other Application An Application End-Point can access its own data or other Application
End-Point's data (provided sufficient authorization) in DECADE End-Point's data (provided sufficient authorization) in DECADE-
servers. Application End-Points may also authorize other End-Points compatible servers. Application End-Points may also authorize other
to store data. If an access is authorized by an Application End- End-Points to store data. If an access is authorized by an
Point, the DECADE Server will provide access. Application End-Point, the Server will provide access.
Note that even if a request is authorized, it may still fail to Even if a request is authorized, it may still fail to complete due to
complete due to insufficient resources by either the requesting insufficient resources by either the requesting Application End-
Application End-Point, the providing Application End-Point, or the Point, the providing Application End-Point, or the Server itself.
DECADE Server itself.
5.2.2. Resource Scheduling 5.2.2. Resource Scheduling
Applications may apply their existing resource sharing policies or Applications apply resource sharing policies or use a custom policy.
use a custom policy for DECADE. DECADE servers perform resource Servers perform resource scheduling according to the resource sharing
scheduling according to the resource sharing policies indicated by policies indicated by Application End-Points as well as configured
Application End-Points as well as configured User Delegations. User Delegations.
5.2.3. Data Store 5.2.3. Data Store
Data from applications may be stored at a DECADE Server. Data can be Data from applications may be stored at a DECADE-compatible Server.
deleted from storage either explicitly or automatically (e.g., after Data can be deleted from storage either explicitly or automatically
a TTL expiration). It may be possible to perform optimizations in (e.g., after a TTL expiration). It may be possible to perform
certain cases, such as avoiding writing temporary data (e.g., live optimizations in certain cases, such as avoiding writing temporary
streaming) to persistent storage, if appropriate storage hints are data (e.g., live streaming) to persistent storage, if appropriate
supported by the SDT. storage hints are supported by the SDT.
5.3. Data Sequencing and Naming 5.3. Data Sequencing and Naming
In order to provide a simple and generic interface, the DECADE Server In order to provide a simple and generic interface, the DECADE-
is only responsible for storing and retrieving individual data compatible Server is only responsible for storing and retrieving
objects. Furthermore, DECADE uses its own simple naming scheme that individual data objects. Furthermore, a DECADE-compatible system
provides uniqueness (with high probability) between data objects, uses its own simple naming scheme that provides uniqueness (with high
even across multiple applications. probability) between data objects, even across multiple applications.
5.3.1. DECADE Data Object Naming Scheme 5.3.1. DECADE Data Object Naming Scheme
The name of a data object is derived from the hash over the data The name of a data object is derived from the hash over the data
object's content (the raw bytes), which is made possible by the fact object's content (the raw bytes), which is made possible by the fact
that DECADE objects are immutable. This scheme multiple appealing that bjects are immutable. This scheme has multiple appealing
properties: properties:
o Simple integrity verification o Simple integrity verification
o Unique names (with high probability) o Unique names (with high probability)
o Application independent, without a new IANA-maintained registry o Application independent, without a new IANA-maintained registry
The DECADE naming scheme also includes a "type" field, the "type" The DECADE-compatible naming scheme also includes a "type" field, the
identifier indicates that the name is the hash of the data object's "type" identifier indicates that the name is the hash of the data
content and the particular hashing algorithm used. This allows object's content and the particular hashing algorithm used. This
DECADE to evolve by either changing the hashing algorithm (e.g., if allows it to evolve by either changing the hashing algorithm (e.g.,
security vulnerabilities with an existing hashing algorithm are if security vulnerabilities with an existing hashing algorithm are
discovered), or moving to a different naming scheme altogether. discovered), or moving to a different naming scheme altogether.
The specific format of the name (e.g., encoding, hash algorithms, The specific format of the name (e.g., encoding, hash algorithms,
etc) is out of scope of this document, and left for protocol etc) is out of scope of this document, and left for protocol
specification. specification.
Another advantage of this scheme is that a DECADE client knows the Another advantage of this scheme is that a DECADE-compatible client
name of a data object before it is completely stored at the DECADE knows the name of a data object before it is completely stored at the
server. This allows for particular optimizations, such as server. This allows for particular optimizations, such as
advertising data object while the data object is being stored, advertising data object while the data object is being stored,
removing store-and-forward delays. For example, a DECADE client A removing store-and-forward delays. For example, a client A may
may simultaneously begin storing an object to a DECADE server, and simultaneously begin storing an object to a server, and advertise
advertise that the object is available to DECADE client B. If it is that the object is available to client B. If it is supported by the
supported by the DECADE server, client B may begin downloading the server, client B may begin downloading the object before A is
object before A is finished storing the object. finished storing the object.
In certain scenarios (e.g., where a DECADE client has limited In certain scenarios (e.g., where a client has limited computational
computational power), it may be advantageous to offload the power), it may be advantageous to offload the computation of a data
computation of a data object's name to the DECADE Server. This object's name to the Server. This possibility is not considered in
possibility is not considered in the current architecture, but may be the current architecture, but may be a topic for future extensions.
a topic for future extensions.
5.3.2. Application Usage 5.3.2. Application Usage
Recall from Section 5.1.1 that an Application typically includes its Recall from Section 5.1.1 that an Application typically includes its
own naming and sequencing scheme. It is important to note that the own naming and sequencing scheme. The DECADE-compatible naming
DECADE naming format does not attempt to replace any naming or format does not attempt to replace any naming or sequencing of data
sequencing of data objects already performed by an Application; objects already performed by an Application; instead, the naming is
instead, the DECADE naming is intended to apply only to data objects intended to apply only to data objects referenced by DECADE-specific
referenced at the DECADE layer. purposes .
DECADE names are not necessarily correlated with the naming or DECADE-compatible names are not necessarily correlated with the
sequencing used by the Application using a DECADE client. The DECADE naming or sequencing used by the Application using a DECADE-
client is expected to maintain a mapping from its own data objects compatible client. The DECADE-compatible client is expected to
and their names to the DECADE data objects and names. Furthermore, maintain a mapping from its own data objects and their names to the
the DECADE naming scheme implies no sequencing or grouping of DECADE-specific data objects and names. Furthermore, the DECADE-
compatible naming scheme implies no sequencing or grouping of
objects, even if this is done at the application layer. objects, even if this is done at the application layer.
Not only does an Application retain its own naming scheme, it may Not only does an Application retain its own naming scheme, it may
also decide the sizes of data objects to be distributed via DECADE. also decide the sizes of data objects to be distributed via the
This is desirable since sizes of data objects may impact Application DECADE-compatible system. This is desirable since sizes of data
performance (e.g., overhead vs. data distribution delay), and the objects may impact Application performance (e.g., overhead vs. data
particular tradeoff is application-dependent. distribution delay), and the particular tradeoff is application-
dependent.
5.3.3. Application Usage Example 5.3.3. Application Usage Example
To illustrate these properties, this section presents multiple To illustrate these properties, this section presents multiple
examples. examples.
5.3.3.1. Application with Fixed-Size Chunks 5.3.3.1. Application with Fixed-Size Chunks
Similar to the example in Section 5.1.1, consider an Application in Similar to the example in Section 5.1.1, consider an Application in
which each individual application-layer segment of data is called a which each individual application-layer segment of data is called a
"chunk" and has a name of the form: "CONTENT_ID:SEQUENCE_NUMBER". "chunk" and has a name of the form: "CONTENT_ID:SEQUENCE_NUMBER".
Furthermore, assume that the application's native protocol uses Furthermore, assume that the application's native protocol uses
chunks of size 16KB. chunks of size 16KB.
Now, assume that this application wishes to make use of DECADE, and Now, assume that this application wishes to store data in DECADE-
assume that it wishes to store data to DECADE servers in data objects compatible servers in data objects of size 64KB. To accomplish this,
of size 64KB. To accomplish this, it can map a sequence of 4 chunks it can map a sequence of 4 chunks into a single object, as shown in
into a single DECADE object, as shown in Figure 5. Figure 5.
Application Chunks Application Chunks
.---------.---------.---------.---------.---------.---------.-------- .---------.---------.---------.---------.---------.---------.--------
| | | | | | | | | | | | | |
| Chunk_0 | Chunk_1 | Chunk_2 | Chunk_3 | Chunk_4 | Chunk_5 | Chunk_6 | Chunk_0 | Chunk_1 | Chunk_2 | Chunk_3 | Chunk_4 | Chunk_5 | Chunk_6
| | | | | | | | | | | | | |
`---------`---------`---------`---------`---------`---------`-------- `---------`---------`---------`---------`---------`---------`--------
DECADE Data Objects DECADE Data Objects
.---------------------------------------.---------------------------- .---------------------------------------.----------------------------
| | | |
| Object_0 | Object_1 | Object_0 | Object_1
| | | |
`---------------------------------------`---------------------------- `---------------------------------------`----------------------------
Figure 5: Mapping Application Chunks to DECADE Data Objects Figure 5: Mapping Application Chunks to DECADE Data Objects
In this example, the Application might maintain a logical mapping In this example, the Application might maintain a logical mapping
that is able to determine the name of a DECADE data object given the that is able to determine the name of a DECADE-compatible data object
chunks contained within that data object. The name might be learned given the chunks contained within that data object. The name might
from either the original source, another endpoint with which the it be learned from either the original source, another endpoint with
is communicating, a tracker, etc. which the it is communicating, a tracker, etc.
It is important to note that as long as the data contained within As long as the data contained within each sequence of chunks is
each sequence of chunks is unique, the corresponding DECADE data unique, the corresponding data objects have unique names. This is
objects have unique names. This is desired, and happens desired, and happens automatically if particular Application segments
automatically if particular Application segments the same stream of the same stream of data in a different way, including different chunk
data in a different way, including different chunk size sizes or sizes or different padding schemes.
different padding schemes.
5.3.3.2. Application with Continuous Streaming Data 5.3.3.2. Application with Continuous Streaming Data
Next, consider an Application whose native protocol retrieves a Next, consider an Application whose native protocol retrieves a
continuous data stream (e.g., an MPEG2 stream) instead of downloading continuous data stream (e.g., an MPEG2 stream) instead of downloading
and redistributing chunks of data. Such an application could segment and redistributing chunks of data. Such an application could segment
the continuous data stream to produce either fixed-sized or variable- the continuous data stream to produce either fixed-sized or variable-
sized DECADE data objects. sized data objects.
Figure 6 shows how a video streaming application might produce Figure 6 shows how a video streaming application might produce
variable-sized DECADE data objects such that each DECADE data object variable-sized data objects such that each data object contains 10
contains 10 seconds of video data. seconds of video data.
Application's Video Stream Application's Video Stream
.-------------------------------------------------------------------- .--------------------------------------------------------------------
| |
| |
| |
`-------------------------------------------------------------------- `--------------------------------------------------------------------
^ ^ ^ ^ ^ ^ ^ ^ ^ ^
| | | | | | | | | |
0 Seconds 10 Seconds 20 Seconds 30 Seconds 40 Seconds 0 Seconds 10 Seconds 20 Seconds 30 Seconds 40 Seconds
skipping to change at page 21, line 31 skipping to change at page 22, line 31
`--------------`--------------`--------------`--------------`-------- `--------------`--------------`--------------`--------------`--------
Figure 6: Mapping a Continuous Data Stream to DECADE Data Objects Figure 6: Mapping a Continuous Data Stream to DECADE Data Objects
Similar to the previous example, the Application might maintain a Similar to the previous example, the Application might maintain a
mapping that is able to determine the name of a DECADE data object mapping that is able to determine the name of a DECADE data object
given the time offset of the video chunk. given the time offset of the video chunk.
5.4. Token-based Authentication and Resource Control 5.4. Token-based Authentication and Resource Control
A primary use case for DECADE is a DECADE Client authorizing other A key feature of a DECADE-compatible system is that a Client can
DECADE Clients to store or retrieve data objects from its DECADE authorize other Clients to store or retrieve data objects from the
storage. To support this, DECADE uses a token-based authentication in-network storage. A token-based authentication scheme is used to
scheme. accomplish this.
In particular, an entity trusted by a DECADE Client generates a Specifically, an entity trusted by a Client generates a digitally-
digitally-signed token with particular properties (see Section 6.1.2 signed token with particular properties (see Section 6.1.2 for
for details). The DECADE Client distributes this token to other details). The Client distributes this token to other Clients which
DECADE Clients which then use the token when sending requests to the then use the token when sending requests to the DECADE-compatible
DECADE Server. Upon receiving a token, the DECADE Server validates Server. Upon receiving a token, the Server validates the signature
the signature and the operation being performed. and the operation being performed.
This is a simple scheme, but has multiple important advantages over This is a simple scheme, but has some important advantages over an
an alternate approach in which a DECADE Client explicitly manipulates alternate approach in which a Client explicitly manipulates an Access
an Access Control List (ACL) associated with each DECADE data object. Control List (ACL) associated with each data object. In particular,
In particular, it has the following advantages when applied to it has the following advantages when applied to DECADE-compliant
DECADE's target applications: target applications:
o Authorization policies are implemented within the Application; an o Authorization policies are implemented within the Application; an
Application explicitly controls when tokens are generated and to Application explicitly controls when tokens are generated and to
whom they are distributed. whom they are distributed.
o Fine-grained access and resource control can be applied to data o Fine-grained access and resource control can be applied to data
objects; see Section 6.1.2 for the list of restrictions that can objects; see Section 6.1.2 for the list of restrictions that can
be enforced with a token. be enforced with a token.
o There is no messaging between a DECADE Client and DECADE Server to o There is no messaging between a Client and Server to manipulate
manipulate data object permissions. This can simplify, in data object permissions. This can simplify, in particular,
particular, Applications which share data objects with many Applications which share data objects with many dynamic peers and
dynamic peers and need to frequently adjust access control need to frequently adjust access control policies attached to data
policies attached to DECADE data objects. objects.
o Tokens can provide anonymous access, in which a DECADE Server does o Tokens can provide anonymous access, in which a Server does not
not need to know the identity of each DECADE Client that accesses need to know the identity of each Client that accesses it. This
it. This enables a DECADE Client to send tokens to DECADE Clients enables a Client to send tokens to Clients in other administrative
in other administrative or security domains, and allow them to or security domains, and allow them to read or write data objects
read or write data objects from its DECADE storage. from its storage.
It is important to note that, in addition to DECADE Clients applying In addition to Clients applying access control policies to data
access control policies to DECADE data objects, the DECADE Server may objects, the Server may be configured to apply additional policies
be configured to apply additional policies based on user, object, based on user, object, geographic location, etc. A Client may thus
geographic location, etc. Defining such policies is out of scope for be denied access even though it possess a valid token.
DECADE, but in such a case, a DECADE Client may be denied access even
though it possess a valid token. Existing protocols (e.g., OAuth [RFC5849]) implement similar referral
mechanisms using tokens. A protocol specification of this
architecture will prefer to use existing mechanisms wherever
possible.
5.5. Discovery 5.5. Discovery
DECADE includes a discovery mechanism through which DECADE clients A DECADE-compatible systme includes a discovery mechanism through
locate an appropriate DECADE Server. [I-D.ietf-decade-reqs] details which clients locate an appropriate Server. [I-D.ietf-decade-reqs]
specific requirements of the discovery mechanism; this section details specific requirements of the discovery mechanism; this
discusses how they relate to other principles outlined in this section discusses how they relate to other principles outlined in
document. this document.
A discovery mechanism allows a DECADE client to determine an IP A discovery mechanism allows a client to determine an IP address or
address or some other identifier that can be resolved to locate the some other identifier that can be resolved to locate the server for
server for which the client will be authorized to generate tokens which the client will be authorized to generate tokens (via DRP).
(via DRP). (Note that the discovery mechanism may also result in an (The discovery mechanism may also result in an error if no such
error if no such DECADE servers can be located.) After discovering servers can be located.) After discovering one or more servers, a
one or more DECADE servers, a DECADE client may distribute load and client may distribute load and requests across them (subject to
requests across them (subject to resource limitations and policies of resource limitations and policies of the servers themselves)
the DECADE servers themselves) according to the policies of the according to the policies of the Application End-Point in which it is
Application End-Point in which it is embedded. embedded.
The particular protocol used for discovery is out of scope of this The particular protocol used for discovery is out of scope of this
document, but any specification will re-use standard protocols document, but any specification will re-use standard protocols
wherever possible. wherever possible.
It is important to note that the discovery mechanism outlined here The discovery mechanism outlined here does not provide the ability to
does not provide the ability to locate arbitrary DECADE servers to locate arbitrary DECADE-compatible servers to which a client might
which a DECADE client might obtain tokens from others. To do so obtain tokens from others. To do so requires application-level
requires application-level knowledge, and it is assumed that this knowledge, and it is assumed that this functionality is implemented
functionality is implemented in the Content Distribution Application, in the Content Distribution Application, or if desired and needed, as
or if desired and needed, as an extension to this DECADE an extension to a DECADE-compatible system.
architecture.
6. DECADE Protocols 6. DECADE Protocols
This section presents the DECADE Resource Protocol (DRP) and the This section presents the DECADE Resource Protocol (DRP) and the
Standard Data Transport (SDT) in terms of abstract protocol Standard Data Transfer (SDT) in terms of abstract protocol
interactions that are intended to mapped to specific protocols. Note interactions that are intended to mapped to specific protocols. In
that while the protocols are logically separate, DRP is specified as general, the DRP/SDT functionality between a DECADE-compatible
being carried through extension fields within an SDT (e.g., HTTP client-server are very similiar to the DRP/SDT functionality between
headers). running between server-server. Any differences are highlighted
below.
The DRP is the protocol used by a DECADE client to configure the The DRP is the protocol used by a DECADE-compatible client to
resources and authorization used to satisfy requests (reading, configure the resources and authorization used to satisfy requests
writing, and management operations concerning DECADE objects) at a (reading, writing, and management operations concerning objects) at a
DECADE server. The SDT is used to send the operations to the DECADE server. The SDT is used to send the operations to the server.
server. Necessary DRP metadata is supplied using mechanisms in the Necessary DRP metadata is supplied using mechanisms in the SDT that
SDT that are provided for extensibility (e.g., additional request are provided for extensibility (e.g., additional request parameters
parameters or extension headers). or extension headers).
6.1. DECADE Resource Protocol (DRP) 6.1. DECADE Resource Protocol (DRP)
DRP provides configuration of access control and resource sharing DRP provides configuration of access control and resource sharing
policies on DECADE servers. A content distribution application, policies on DECADE-compatible servers. A content distribution
e.g., a live P2P streaming session, MAY employ several DECADE application, e.g., a live P2P streaming session, may have permission
servers, for instance, servers in different operator domains, and DRP to manage data at several servers, for instance, servers in different
allows one instance of such an application, e.g., an application operator domains, and DRP allows one instance of such an application,
endpoint, to apply access control and resource sharing policies on e.g., an application endpoint, to apply access control and resource
each of them. sharing policies on each of them.
6.1.1. Controlled Resources 6.1.1. Controlled Resources
On a single DECADE server, the following resources may be managed: On a single DECADE-compatible server, the following resources may be
managed:
communication resources: DECADE servers have limited communication communication resources: Servers have limited communication
resources in terms of bandwidth (upload/download) but also in resources in terms of bandwidth (upload/download) but also in
terms of number of connected clients (connections) at a time. terms of number of connected clients (connections) at a time.
storage resources: DECADE servers have limited storage resources. storage resources: Servers have limited storage resources.
6.1.2. Access and Resource Control Token 6.1.2. Access and Resource Control Token
A token includes the following fields: A token includes the following information:
Permitted operations (e.g., read, write) Permitted operations (e.g., read, write)
Permitted objects (e.g., names of data objects that may be read or Permitted objects (e.g., names of data objects that may be read or
written) written)
Permitted clients (e.g., as indicated by IP address or other
identifier) that may use the token
Expiration time Expiration time
Priority for bandwidth given to requested operation (e.g., a Priority for bandwidth given to requested operation (e.g., a
weight used in a weighted bandwidth sharing scheme) weight used in a weighted bandwidth sharing scheme)
Amount of data that may be read or written Amount of data that may be read or written
The particular format for the token is out of scope of this document. The particular format for the token is out of scope of this document.
The tokens are generated by a trusted entity at the request of a The tokens are generated by a trusted entity at the request of a
DECADE Client. It is out of scope of this document to identify which DECADE-compatible Client. It is out of scope of this document to
entity serves this purpose, but examples include the DECADE Client identify which entity serves this purpose, but examples include the
itself, a DECADE Server trusted by the DECADE Client, or another Client itself, a Server trusted by the Client, or another server
server managed by a Storage Provider trusted by the DECADE Client. managed by a Storage Provider trusted by the Client.
Upon generating a token, a DECADE Client may distribute it to another Upon generating a token, a Client may distribute it to another Client
DECADE Client (e.g., via their native Application protocol). The (e.g., via their native Application protocol). The receiving Client
receiving DECADE Client may then connect to the sending DECADE may then connect to the sending Client's Server and perform any
Client's DECADE Server and perform any operation permitted by the operation permitted by the token. The token must be sent along with
token. The token must be sent along with the operation. The DECADE the operation. The Server validates the token to identify the Client
Server validates the token to identify the DECADE Client that issued that issued it and whether the requested operation is permitted by
it and whether the requested operation is permitted by the contents the contents of the token. If the token is successfully validated,
of the token. If the token is successfully validated, the DECADE the Server applies the resource control policies indicated in the
Server applies the resource control policies indicated in the token token while performing the operation.
while performing the operation.
It is possible for DRP to allow tokens to apply to a batch of Tokens include a unique identifier to allow a Server to detect when a
operations to reduce communication overhead required between DECADE token is used multiple times.
Clients.
DRP may also define tokens to include a unique identifier to allow a It is possible for DRP to allow tokens to apply to a batch of
DECADE Server to detect when a token is used multiple times. operations to reduce communication overhead required between Clients.
6.1.3. Status Information 6.1.3. Status Information
DRP provides a request service for status information that DECADE DRP provides a request service for status information that clients
clients can use to request information from a DECADE server. can use to request information from a server.
status information per application context on a specific server: status information per application context on a specific server:
Access to such status information requires client authorization, Access to such status information requires client authorization,
i.e., DECADE clients need to be authorized to access status i.e., clients need to be authorized to access status information
information for a specific application context. This for a specific application context. This authorization (and the
authorization (and the mapping to application contexts) is based mapping to application contexts) is based on the user delegation
on the user delegation concept as described in Section 4.5. The concept as described in Section 4.6. The following status
following status information elements can be obtained: information elements can be obtained:
* list of associated objects (with properties) * list of associated objects (with properties)
* resources used/available * resources used/available
* list of servers to which objects have been distributed (in a * list of servers to which objects have been distributed (in a
certain time-frame) certain time-frame)
* list of clients to which objects have been distributed (in a * list of clients to which objects have been distributed (in a
certain time-frame) certain time-frame)
For the list of servers/clients to which objects have been For the list of servers/clients to which objects have been
distributed to, the DECADE server can decide on time bounds for distributed to, the server can decide on time bounds for which
which this information is stored and specify the corresponding this information is stored and specify the corresponding time
time frame in the response to such requests. Some of this frame in the response to such requests. Some of this information
information can be used for accounting purposes, e.g., the list of can be used for accounting purposes, e.g., the list of clients to
clients to which objects have been distributed. which objects have been distributed.
access information per application context on a specific server: access information per application context on a specific server:
Access information can be provided for accounting purposes, for Access information can be provided for accounting purposes, for
example, when application service providers are interested to example, when application service providers are interested to
maintain access statistics for resources and/or to perform maintain access statistics for resources and/or to perform
accounting per user. Again, access to such information requires accounting per user. Again, access to such information requires
client authorization based on the user delegation concept as client authorization based on the user delegation concept as
described in Section 4.5. The following access information described in Section 4.6. The following access information
elements can be requested: elements can be requested:
* what objects have been accessed how many times * what objects have been accessed how many times
* access tokens that a server as seen for a given object * access tokens that a server as seen for a given object
The DECADE server can decide on time bounds for which this The server can decide on time bounds for which this information is
information is stored and specify the corresponding time frame in stored and specify the corresponding time frame in the response to
the response to such requests. such requests.
6.1.4. Object Attributes 6.1.4. Object Attributes
Objects that are stored on a DECADE server may have associated Objects that are stored on a DECADE-compatible server may have
attributes (in addition to the object identifier and the actual associated attributes (in addition to the object identifier and the
content) that relate to the data storage and its management. These actual content) that relate to the data storage and its management.
attributes may be used by the DECADE server (and possibly the These attributes may be used by the server (and possibly the
underlying storage system) to perform specialized processing or underlying storage system) to perform specialized processing or
handling for the data object, or to attach related DECADE server or handling for the data object, or to attach related server or storage-
storage-layer properties to the data object. These attributes have a layer properties to the data object. These attributes have a scope
scope local to a DECADE server. In particular, these attributes are local to a server. In particular, these attributes are not applied
not applied to a DECADE server or client to which a data object is to a server or client to which a data object is copied.
copied.
Depending on authorization, DECADE clients may get or set such Depending on authorization, clients may get or set such attributes.
attributes. This authorization (and the mapping to application This authorization (and the mapping to application contexts) is based
contexts) is based on the user delegation concept as described in on the user delegation concept as described in Section 4.6. The
Section 4.5. The DECADE architecture does not limit the set of architecture does not limit the set of permissible attributes, but
permissible attributes, but rather specifies a set of baseline rather specifies a set of baseline attributes that should be
attributes that SHOULD be supported by implementations. supported by implementations.
Suggested attributes are the following: Suggested attributes are the following:
TTL: TTL of the object as an absolute time value Expiration Time: Time at which the object may be deleted
object size: in bytes object size: in bytes
MIME type Media type
access statistics: how often the object has been accessed (and what access statistics: how often the object has been accessed (and what
tokens have been used) tokens have been used)
It is important to note that the Object Attributes defined here are The Object Attributes defined here are distinct from application
distinct from application metadata (see Section 4.1). Application metadata (see Section 4.1). Application metadata is custom
metadata is custom information that an application may wish to information that an application may wish to associate with a data
associate with a data object to understand its semantic meaning object to understand its semantic meaning (e.g., whether it is video
(e.g., whether it is video and/or audio, its playback length in time, and/or audio, its playback length in time, or its index in a stream).
or its index in a stream). If an application wishes to store such If an application wishes to store such metadata persistently, it can
metadata persistently within DECADE, it can be stored within data be stored within data objects themselves.
objects themselves.
6.2. Standard Data Transport (SDT)
A DECADE server provide a data access interface, and SDT is used to 6.2. Standard Data Transfer (SDT)
write objects to a server and to read (download) objects from a
server. Semantically, SDT is a client-server protocol, i.e., the
DECADE server always responds to client requests.
An SDT used in DECADE SHOULD offer a transport mode that provides A DECADE-compatible server provide a data access interface, and SDT
confidentiality and integrity. is used to write objects to a server and to read (download) objects
from a server. Semantically, SDT is a client-server protocol, i.e.,
the server always responds to client requests.
6.2.1. Writing/Uploading Objects 6.2.1. Writing/Uploading Objects
To write an object, a client first generates the object's name (see To write an object, a client first generates the object's name (see
Section 5.3), and then uploads the object to a DECADE server and Section 5.3), and then uploads the object to a server and supplies
supplies the generated name. The name can be used to access the generated name. The name can be used to access (download) the
(download) the object later, e.g., the client can pass the name as a object later, e.g., the client can pass the name as a reference to
reference to other client that can then refer to the object. other client that can then refer to the object.
DECADE objects can be self-contained objects such as multimedia Objects can be self-contained objects such as multimedia resources,
resources, files etc., but also chunks, such as chunks of a P2P files etc., but also chunks, such as chunks of a P2P distribution
distribution protocol that can be part of a containing object or a protocol that can be part of a containing object or a stream.
stream.
A server MAY accept download requests for an object that is still A server may accept download requests for an object that is still
being uploaded. being uploaded.
The application that originates the objects MUST generate DECADE The application that originates the objects generates DECADE-
object names according to the naming specification in Section 5.3. compatible object names according to the naming specification in
The naming scheme provides that the name is unique. DECADE clients Section 5.3. The naming scheme provides that the name is unique.
(as parts of application entities) upload a named object to a server, Clients (as parts of application entities) upload a named object to a
and a DECADE server MUST NOT change the name. It MUST be possible server. A server may verify the integrity and other security
for downloading clients, to access the object using the original
name. A DECADE server MAY verify the integrity and other security
properties of uploaded objects. properties of uploaded objects.
In the following we provide an abstract specification of the upload In the following we provide an abstract specification of the upload
operation that we name 'PUT METHOD'. See Appendix A.1 for an example operation that we name 'PUT METHOD'.
how this could be mapped to HTTP.
Method PUT: Method PUT:
Parameters: Parameters:
NAME: The naming of the object according to Section 5.3 NAME: The naming of the object according to Section 5.3
OBJECT: The object itself. The protocol MUST provide transparent OBJECT: The object itself.
binary object transport.
Description: The PUT method is used by a DECADE client to upload an Description: The PUT method is used by a DECADE-compatible client to
object with an associated name 'NAME' to a DECADE server. upload an object with an associated name 'NAME' to a DECADE-
compatible server.
RESPONSES: The DECADE server MUST respond with one the following RESPONSES: The server responds with one the following response
response messages: messages:
CREATED: The object has been uploaded successfully and is now CREATED: The object has been uploaded successfully and is now
available under the specified name. available under the specified name.
ERRORs: ERRORs:
VALIDATION_FAILED: The contents of the data object received by VALIDATION_FAILED: The contents of the data object received by
the DECADE server did not match the provided name (i.e., the server did not match the provided name (i.e., hash
hash validation failed). validation failed).
PERMISSION_DENIED: The DECADE client lacked sufficient PERMISSION_DENIED: The client lacked sufficient permission to
permission to store the object. store the object.
Specifics regarding error handling, including additional error Specifics regarding error handling, including additional error
conditions, precedence for returned errors and its relation conditions, precedence for returned errors and its relation
with server policy, are deferred to eventual protocol with server policy, are deferred to eventual protocol
specification. specification.
6.2.2. Downloading Objects 6.2.2. Downloading Objects
A DECADE client can request named objects from a DECADE server. In A client can request named objects from a server. In the following,
the following, we provide an abstract specification of the download we provide an abstract specification of the download operation that
operation that we name 'GET METHOD'. See Appendix A.1 for an example we name 'GET METHOD'.
how this could be mapped to HTTP.
Method GET: Method GET:
Parameters: Parameters:
NAME: The naming of the object according to Section 5.3. NAME: The naming of the object according to Section 5.3.
Description: The GET method is used by a DECADE client to download Description: The GET method is used by a client to download an
an object with an associated name 'NAME' from a DECADE server. object with an associated name 'NAME' from a server.
RESPONSES: The DECADE server MUST respond with one the following RESPONSES: The server responds with one the following response
response messages: messages:
OK: The request has succeeded, and an entity corresponding to the OK: The request has succeeded, and an entity corresponding to the
requested resource is sent in the response. requested resource is sent in the response.
ERRORs: ERRORs:
NOT_FOUND: The DECADE server has not found anything matching NOT_FOUND: The server has not found anything matching the
the request object name. request object name.
PERMISSION_DENIED: The DECADE client lacked sufficient PERMISSION_DENIED: The client lacked sufficient permission to
permission to read the object. read the object.
NOT_AVAILABLE: The data object exists but is currently NOT_AVAILABLE: The data object exists but is currently
unavailable for download (e.g., due to server policy). unavailable for download (e.g., due to server policy).
Specifics regarding error handling, including additional error Specifics regarding error handling, including additional error
conditions (e.g. overload), precedence for returned errors and conditions (e.g. overload), precedence for returned errors and
its relation with server policy, are defered to eventual its relation with server policy, are deferred to eventual
protocol specification. protocol specification.
7. Server-to-Server Protocols 6.3. Server-to-Server Protocols
An important feature of DECADE is the capability for one DECADE An important feature of a DECADE-compatible system is the capability
server to directly download objects from another DECADE server. This for one server to directly download objects from another server.
capability allows Applications to directly replicate data objects This capability allows Applications to directly replicate data
between servers without requiring end-hosts to use uplink capacity to objects between servers without requiring end-hosts to use uplink
upload data objects to a different DECADE server. capacity to upload data objects to a different server.
To support this functionality, DECADE uses the DRP and SDT to support DRP and SDT support operations directly between servers. Servers are
operations directly between servers. DECADE servers are not assumed not assumed to trust each other nor are configured to do so. All
to trust each other nor are configured to do so. All data operations data operations are performed on behalf of clients via explicit
are performed on behalf of DECADE clients via explicit instruction. instruction. However, the objects being processed do not necessarily
Note, however, that the objects being processed do not necessarily have to originate or terminate at the client (i.e. the object may be
have to originate or terminate at the DECADE client (i.e. the object limited to being exchanged between servers even if the instruction is
may be limited to being exchanged between DECADE servers even if the triggered by the client). clients thus must be able to indicate to a
instruction is triggered by the client). DECADE clients thus must be server the following additional parameters:
able to indicate to a DECADE server the following additional
parameters:
o which remote DECADE server(s) to access; o which remote server(s) to access;
o the operation to be performed (e.g. PUT, GET); and o the operation to be performed (e.g. PUT, GET); and
o Credentials indicating permission to perform the operation at the o Credentials indicating permission to perform the operation at the
remote DECADE server. remote server.
In this way, a DECADE server acts as a proxy for a DECADE client, and In this way, a server acts as a proxy for a client, and a client may
a DECADE client may instantiate requests via that proxy. The instantiate requests via that proxy. The operations are performed as
operations are performed as if the original requester had its own if the original requester had its own client co-located with the
DECADE client co-located with the DECADE server. It is this mode of server. It is this mode of operation that provides substantial
operation that provides substantial savings in uplink capacity. Note savings in uplink capacity. This mode of operation may also be
that this mode of operation may also be triggered by an triggered by an administrative/management application outside the
administrative/management application outside the DECADE
architecture. architecture.
7.1. Operational Overview 6.3.1. Operational Overview
DECADE's server-to-server support is focused on reading and writing Server-to-server support is focused on reading and writing data
data objects between DECADE servers. A DECADE GET or PUT request MAY objects between servers. A DECADE-compatible GET or PUT request may
supply the following additional parameters: supply the following additional parameters:
REMOTE_SERVER: Address of the remote DECADE server. The format of REMOTE_SERVER: Address of the remote server. The format of the
the address is out-of-scope of this document. address is out-of-scope of this document.
REMOTE_USER: The account at the remote server from which to retrieve REMOTE_USER: The account at the remote server from which to retrieve
the object (for a GET), or in which the object is to be stored the object (for a GET), or in which the object is to be stored
(for a PUT). (for a PUT).
TOKEN: Credentials to be used at the remote server. TOKEN: Credentials to be used at the remote server.
These parameters are used by the DECADE server to instantiate a These parameters are used by the server to instantiate a request to
request to the specified remote server. It is assumed that the data the specified remote server. It is assumed that the data object
object referred to at the remote server is the same as the original referred to at the remote server is the same as the original request.
request. It is also assumed that the operation performed at the Object attributes (see Section 6.1.4) may also be specified in the
remote server is the same as the operation in the original request. request to the remote server.
Note that object attributes (see Section 6.1.4) may also be specified
in the request to the remote server.
Note that when a DECADE client invokes a request a DECADE server with
these additional parameters, it is giving the DECADE server
permission to act (proxy) on its behalf. Thus, it would be wise for
the supplied token to have narrow privileges (e.g., limited to only
the necessary data objects) or validity time (e.g., a small
expiration time).
In the case of a GET operation, the DECADE server is to retrieve the
data object from the remote server using the specified credentials
(via a GET request to the remote server), and then optionally return
the object to a client. In the case of a PUT operation, the DECADE
server is to store the object to the remote server using the
specified credentials (via a PUT request to the remote server). The
object may optionally be uploaded from the client or may already
exist at the proxying server.
8. Potential Optimizations
As suggestions for the protocol design and eventual implementations,
we discuss particular optimizations that are enabled by the DECADE
Architecture discussed in this document.
8.1. Pipelining to Avoid Store-and-Forward Delays
A DECADE server may choose to not fully store an object before
beginning to serve it. For example, when serving a GET request,
instead of waiting for the complete data to arrive from a remote
server or DECADE client, a DECADE server may forward received data
bytes as they come in. This pipelining mode reduces store-and-
forward delays, which could be substantial for large objects. A
similar behavior could be used for PUT.
8.2. Deduplication
A common concern amongst Storage Providers is the total volume of
data that needs to be stored. An optimization frequently applied in
existing storage systems is de-duplication, which attempts to avoid
storing identical data multiple times. A DECADE Server
implementation may internally perform de-duplication of data on disk.
The DECADE architecture enables additional forms of de-duplication.
Note that these techniques may impact protocol design. Discussions
of whether or not they should be adopted is out of the scope of this
document.
8.2.1. Traffic De-duplication
8.2.1.1. Rationale When a client sends a request to a server with these additional
parameters, it is giving the server permission to act (proxy) on its
behalf. Thus, it would be prudent for the supplied token to have
narrow privileges (e.g., limited to only the necessary data objects)
or validity time (e.g., a small expiration time).
When a DECADE client (A) indicates its DECADE account on a DECADE In the case of a GET operation, the server is to retrieve the data
server (S) to fetch an object from a remote entity (R) (a DECADE object from the remote server using the specified credentials (via a
server or DECADE client) and if the object is already stored locally GET request to the remote server), and then optionally return the
in S, S may perform Traffic De-duplication. This means that S does object to a client. In the case of a PUT operation, the server is to
not download the object from R, in order to save network traffic. In store the object to the remote server using the specified credentials
particular, S performs a challenge to make sure that the remote (via a PUT request to the remote server). The object may optionally
entity R actually has the object and then replies with its local be uploaded from the client or may already exist at the proxying
object copy directly. server.
8.2.1.2. An Example 7. Security Considerations
As shown in Figure 7, without Traffic De-duplication, unnecessary In general, the security considerations mentioned in
transfer of an object from R to S may happen, if the server S already [I-D.ietf-decade-problem-statement] apply to this document as well.
has the object requested by A. If Traffic De-duplication is enabled,
S only needs to challenge R to verify that it does have the data to
avoid data-stealing attacks.
A S R A DECADE-compatible system provides a distributed storage service for
+----------+ obj req +------------+ obj req +----------+ content distribution and similar applications. The system consists
| DECADE |=========>| A's |==========>| Remote | of servers and clients that use these servers to upload data objects,
| CLIENT |<=========| Account |<==========| Entity | to request distribution of data objects, and to download data
+----------+ obj rsp +------------+ obj rsp +----------+ objects. Such a system is employed in an overall application context
-- for example in a P2P content distribution application, and it is
generally expected that DECADE-compatible clients take part in
application-specific communication sessions.
(a) Without Traffic De-duplication The security considerations here focus on threats related to the
DECADE-compatible system and its communication services, i.e., the
DRP/SDT protocols that have been described in an abstract fashion in
this document.
A S R 7.1. Threat: System Denial of Service Attacks
+----------+ obj req +------------+ challenge +----------+
| DECADE |=========>| A's |---------->| Remote |
| CLIENT |<=========| Account |<----------| Entity |
+----------+ obj rsp +------------+ obj hash +----------+
(b) With Traffic De-duplication A DECADE-compatible network of servers may be used to distribute data
objects from one client to a set of servers using the server-to-
server communication feature that a client can request when uploading
an object. Multiple clients uploading many objects at different
servers at the same time and requesting server-to-server distribution
for them could thus mount massive distributed denial of service
(DDOS) attacks, overloading a network of servers.
Figure 7 This threat is addressed by its access control and resource control
framework. Servers can require application endpoints to be
authorized to store and to download objects, and application
endpoints can delegate authorization to other application endpoints
using the token mechanism.
8.2.1.3. HTTP Compatibility of Challenge Of course the effective security of this approach depends on the
strength of the token mechanism. See below for a discussion of this
and related communication security threats.
How to integrate traffic de-duplication with HTTP is shown in Denial of Service Attacks against a single server (directing many
Appendix A.1.3. requests to that server) may still lead to considerable load for
processing requests and invalidating tokens. A SDT therefore MUST
provide a redirection mechanism as described as a requirement in
[I-D.ietf-decade-reqs].
8.2.2. Cross-Server Storage De-duplication 7.2. DECADE Protocol Security Threats
The same object might be uploaded multiple times to different DECADE 7.2.1. Threat: Authorization Mechanisms Compromised
servers. For storage efficiency, storage providers may desire that a
single object be stored on one or a few servers. They might
implement an internal mechanism to achieve the goal, for example, by
redirecting requests to proper servers. DECADE supports the
redirection of DECADE client requests to support further cross-server
storage de-duplication.
9. Security Considerations A DECADE-compatible system does not require application endpoints to
authenticate in order to access a server for downloading objects,
since authorization is not based on endpoint or user identities but
on the delegation-based authorization mechanism. Hence, most
protocol security threats are related to the authorization scheme.
In general, the security considerations mentioned in The security of the token mechanism depends on the strength of the
[I-D.ietf-decade-problem-statement] apply to this document as well. token mechanism and on the secrecy of the tokens. A token can
represent authorization to store a certain amount of data, to
download certain objects, to download a certain amount of data per
time etc. If it is possible for an attacker to guess, construct or
simply obtain tokens, the integrity of the data maintained by the
servers, or at least the affected application context on servers, is
compromised.
In addition, it should be noted that the token-based approach This is a general security threat that applies to authorization
Section 5.4 provides authorization through token delegation. The delegation schemes. Specifications of existing delegation schemes
strength of this authorization depends on several factors: such as OAuth [RFC5849] discuss these general threats in detail. We
can say that the DRP has to specify appropriate algorithms for token
generation. Moreover, authorization tokens should have a limited
validity period that should be specified by the application. Token
confidentiality should be provided by application protocols that
carry tokens, and the SDT and DRP should provide secure
(confidential) communication modes.
1. the uniqueness of tokens: tokens should be constructed in a way 7.2.2. Threat: Object Spoofing
that minimize the possibilities for collisions;
2. validity of tokens: applications/users should not re-use tokens; In a DECADE-compliant system, an application endpoint is referring
and other application endpoints to servers to download a specified data
objects. An attacker could "inject" a faked version of the object
into this process, so that the downloading endpoint effectively
receives a different object (compared to what the uploading endpoint
provided). As result, the downloading endpoint believes that is has
received an object that corresponds to the name it was provided
earlier, whereas in fact it is a faked object. Corresponding attacks
could be mounted against the application protocol (that is used for
referring other endpoints to servers), servers themselves (and their
storage sub-systems), and the SDT by which the object is uploaded,
distributed and downloaded.
3. secrecy of tokens: if tokens are compromised to unauthorized A DECADE-compliant systems fundamental mechanism against object
entities, access control for the associated resources cannot be spoofing is name-content binding validation, i.e., the ability of a
provided. receiver to check whether the name he was provided and that he used
to request an object, actually corresponds to the bits he received.
As described above, this allows for different forms of name-content
binding, for example using content hashes, with different hash
functions (different algorithms, different digest lengths). For
those application scenarios where content hashes are not applicable
(for example live-streaming) other forms of name-content binding can
be used (see Section 5.3. This flexibility also addresses
cryptographic algorithm evolvability: hash functions may get
deprecated, better alternatives may be invented etc., so that
applications can choose appropriate mechanisms meeting their security
requirements.
Depending on the specific application, DECADE can be used to access DECADE-compliant servers MAY perform name-content binding validation
confidential information. Hence DECADE implementations SHOULD on stored objects, but application endpoints MUST NOT rely on that.
provide a secure transport mode that allows for encryption. In other forms: application endpoints SHOULD perform name-content
binding validation on received objects.
10. IANA Considerations 8. IANA Considerations
This document does not have any IANA considerations. This document does not have any IANA considerations.
11. Acknowledgments 9. Acknowledgments
We thank the following people for their contributions to this We thank the following people for their contributions to this
document: document:
David Bryan David Bryan
Hongqiang (Harry) Liu
Yingjie Gu Yingjie Gu
David McDysan David McDysan
Borje Ohlman Borje Ohlman
Haibin Song Haibin Song
Martin Stiemerling Martin Stiemerling
skipping to change at page 33, line 35 skipping to change at page 34, line 4
David McDysan David McDysan
Borje Ohlman Borje Ohlman
Haibin Song Haibin Song
Martin Stiemerling Martin Stiemerling
Richard Woundy Richard Woundy
Ning Zong Ning Zong
12. References 10. References
12.1. Normative References 10.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
12.2. Informative References 10.2. Informative References
[RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,
Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext
Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999. Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.
[RFC3744] Clemm, G., Reschke, J., Sedlar, E., and J. Whitehead, "Web [RFC3744] Clemm, G., Reschke, J., Sedlar, E., and J. Whitehead, "Web
Distributed Authoring and Versioning (WebDAV) Distributed Authoring and Versioning (WebDAV)
Access Control Protocol", RFC 3744, May 2004. Access Control Protocol", RFC 3744, May 2004.
[RFC4288] Freed, N. and J. Klensin, "Media Type Specifications and
Registration Procedures", BCP 13, RFC 4288, December 2005.
[RFC4331] Korver, B. and L. Dusseault, "Quota and Size Properties fo [RFC4331] Korver, B. and L. Dusseault, "Quota and Size Properties fo
r Distributed Authoring and Versioning (DAV) Collections", r Distributed Authoring and Versioning (DAV) Collections",
RFC 4331, February 2006. RFC 4331, February 2006.
[RFC4709] Reschke, J., "Mounting Web Distributed Authoring and [RFC4709] Reschke, J., "Mounting Web Distributed Authoring and
Versioning (WebDAV) Servers", RFC 4709, October 2006. Versioning (WebDAV) Servers", RFC 4709, October 2006.
[RFC4918] Dusseault, L., "HTTP Extensions for Web Distributed [RFC4918] Dusseault, L., "HTTP Extensions for Web Distributed
Authoring and Versioning (WebDAV)", RFC 4918, June 2007. Authoring and Versioning (WebDAV)", RFC 4918, June 2007.
[RFC5661] Shepler, S., Eisler, M., and D. Noveck, "Network File [RFC5661] Shepler, S., Eisler, M., and D. Noveck, "Network File
System (NFS) Version 4 Minor Version 1 Protocol", System (NFS) Version 4 Minor Version 1 Protocol",
RFC 5661, January 2010. RFC 5661, January 2010.
[RFC5849] Hammer-Lahav, E., "The OAuth 1.0 Protocol", RFC 5849,
April 2010.
[RFC6392] Alimi, R., Rahman, A., and Y. Yang, "A Survey of In- [RFC6392] Alimi, R., Rahman, A., and Y. Yang, "A Survey of In-
Network Storage Systems", RFC 6392, October 2011. Network Storage Systems", RFC 6392, October 2011.
[I-D.ietf-decade-problem-statement] [I-D.ietf-decade-problem-statement]
Song, H., Zong, N., Yang, Y., and R. Alimi, "DECoupled Song, H., Zong, N., Yang, Y., and R. Alimi, "DECoupled
Application Data Enroute (DECADE) Problem Statement", Application Data Enroute (DECADE) Problem Statement",
draft-ietf-decade-problem-statement-04 (work in progress), draft-ietf-decade-problem-statement-05 (work in progress),
October 2011. February 2012.
[I-D.ietf-decade-reqs] [I-D.ietf-decade-reqs]
Yingjie, G., Bryan, D., Yang, Y., and R. Alimi, "DECADE Yingjie, G., Bryan, D., Yang, Y., and R. Alimi, "DECADE
Requirements", draft-ietf-decade-reqs-04 (work in Requirements", draft-ietf-decade-reqs-05 (work in
progress), September 2011. progress), October 2011.
[GoogleStorageDevGuide] [GoogleStorageDevGuide]
"Google Storage Developer Guide", <http://code.google.com/ "Google Storage Developer Guide", <http://code.google.com/
apis/storage/docs/developer-guide.html>. apis/storage/docs/developer-guide.html>.
[OpenFlow]
"OpenFlow Organization", <http://www.openflow.org/>.
[GoogleFileSystem] [GoogleFileSystem]
Ghemawat, S., Gobioff, H., and S. Leung, "The Google File Ghemawat, S., Gobioff, H., and S. Leung, "The Google File
System", SOSP 2003, October 2003. System", SOSP 2003, October 2003.
[CDMI] "CDMI", <http://www.snia.org/cdmi>. Appendix A. In-Network Storage Components Mapped to DECADE Architecture
Appendix A. Appendix: Evaluation of Some Candidate Existing Protocols
for DECADE DRP and SDT
In this section we evaluate how well the abstract protocol
interactions specified in Section 6 for DECADE DRP and SDT can be
fulfilled by existing protocols such as HTTP, WEBDAV, and CDMI.
A.1. HTTP
HTTP [RFC2616] is a key protocol for the Internet in general and
especially for the World Wide Web. HTTP is a request-response
protocol. A typical transaction involves a client (e.g. web browser)
requesting content (resources) from a web server. Another example is
when a client stores or deletes content from a server.
A.1.1. HTTP Support for DECADE Resource Protocol Primitives
DRP provides configuration of access control and resource sharing
policies on DECADE servers.
A.1.1.1. Access Control Primitives
Access control requires mechanisms for defining the access policies
for the server, and then checking the authorization of a user before
it stores or retrieves content. HTTP supports a rudimentary access
control via "HTTP Secure" (HTTPS). HTTPS is a combination of HTTP
with SSL/TLS. The main use of HTTPS is to authenticate the server
and encrypt all traffic between the client and the server. There is
also a mode to support client authentication though this is less
frequently used.
A.1.1.2. Communication Resource Controls Primitives
Communication resources include bandwidth (upload/download) and
number of simultaneous connected clients (connections). HTTP
supports bandwidth control indirectly through "persistent" HTTP
connections. Persistent HTTP connections allows a client to keep
open the underlying TCP connection to the server to allow streaming
and pipelining (multiple simultaneous requests for a given client).
HTTP does not define protocol operation to allow limiting the
communication resources to a client. However servers typically
perform this function via implementation algorithms.
A.1.1.3. Storage Resource Control Primitives
Storage resources include amount of memory and lifetime of storage.
HTTP does not allow direct control of storage at the server end
point. However HTTP supports caching at intermediate points such as
a web proxy. For this purpose, HTTP defines cache control mechanisms
that define how long and in what situations the intermediate point
may store and use the content.
A.1.2. HTTP Support for DECADE Standard Data Transport Protocol
Primitives
SDT is used to write objects and read (download) objects from a
DECADE server. The object can be either a self-contained object such
as a multimedia file or a chunk from a P2P system.
A.1.2.1. Writing Primitives
Writing involves uploading objects to the server. HTTP supports two
methods of writing called PUT and POST. In HTTP the object is called
a resource and is identified by a URI. PUT uploads a resource to a
specific location on the server. POST, on the other hand, submits
the object to the server and the server decides whether to update an
existing resource or to create a new resource.
For DECADE, the choice of whether to use PUT or POST will be
influenced by which entity is responsible for the naming. If the
client performs the naming, then PUT is appropriate. If the server
performs the naming, then POST should be used (to allow the server to
define the URI).
A.1.2.2. Downloading Primitives
Downloading involves fetching of an object from the server. HTTP
supports downloading through the GET and HEAD methods. GET fetches a
specific resource as identified by the URL. HEAD is similar but only
fetches the metadata ("header") associated with the resource but not
the resource itself.
A.1.3. Traffic De-duplication Primitives
To challenge a remote entity for an object, the DECADE server should
provide a seed number, which is generated by the server randomly, and
ask the remote entity to return a hash calculated from the seed
number and the content of the object. The server may also specify
the hash function which the remote entity should use. HTTP supports
the challenge message through the GET methods. The message type
("challenge"), the seed number and the hash function name are put in
URL. In the reply, the hash is sent in an ETAG header.
A.1.4. Other Operations
HTTP supports deleting of content on the server through the DELETE
method.
A.1.5. Conclusions
HTTP can provide a rudimentary DRP and SDT for some aspects of
DECADE, but will not be able to satisfy all the DECADE requirements.
For example, HTTP does not provide a complete access control
mechanism, nor does it support storage resource controls at the end
point server.
It is possible, however, to envision combining HTTP with a custom
suite of other protocols to fulfill most of the DECADE requirements
for DRP and SDT. For example, Google Storage for Developers is built
using HTTP (with extensive proprietary extensions such as custom HTTP
headers). Google Storage also uses OAUTH 2.0 (for access control) in
combination with HTTP [GoogleStorageDevGuide].
A.2. WEBDAV
WebDAV [RFC4918] is a protocol for enhanced Web content creation and
management. It was developed as an extension to HTTP Appendix A.1.
WebDAV supports traditional operations for reading/writing from
storage, as well as more advanced features such as locking and
namespace management which are important when multiple users
collaborate to author or edit a set of documents. HTTP is a subset
of WebDAV functionality. Therefore, all the points noted above in
Appendix A.1 apply implicitly to WebDAV as well.
A.2.1. WEBDAV Support for DECADE Resource Protocol Primitives
DRP provides configuration of access control and resource sharing
policies on DECADE servers.
A.2.1.1. Access Control Primitives
Access control requires mechanisms for defining the access policies
for the server, and then checking the authorization of a user before
it stores or retrieves content. WebDAV has an Access Control
Protocol defined in [RFC3744].
The goal of WebDAV access control is to provide an interoperable
mechanism for handling discretionary access control for content and
metadata managed by WebDAV servers. WebDAV defines an Access Control
List (ACL) per resource. An ACL contains a set of Access Control
Entries (ACEs), where each ACE specifies a principal (i.e. user or
group of users) and a set of privileges that are granted to that
principal. When a principal tries to perform an operation on that
resource, the server evaluates the ACEs in the ACL to determine if
the principal has permission for that operation.
WebDAV also requires that an authentication mechanism be available
for the server to validate the identity of a principal. As a
minimum, all WebDAV compliant implementations are required to support
HTTP Digest Authentication.
A.2.1.2. Communication Resource Controls Primitives
Communications resources include bandwidth (upload/download) and
number of simultaneous connected clients (connections). WebDAV
supports communication resource control as described in
Appendix A.1.1.2.
A.2.1.3. Storage Resource Control Primitives
Storage resources include amount of memory and lifetime of storage.
WebDAV supports the concept of properties (which are metadata for a
resource). A property is either "live" or "dead". Live properties
include cases where a) the value of a property is protected and
maintained by the server, and b) the value of the property is
maintained by the client, but the server performs syntax checking on
submitted values. A dead property has its syntax and semantics
enforced by the client; the server merely records the value of the
property.
WebDAV supports a list of standardized properties [RFC4918] that are
useful for storage resource control. These include the self-
explanatory "creationdate", and "getcontentlength" properties. There
is also an operation called PROPFIND to retrieve all the properties
defined for the requested URI.
WebDAV also has a Quota and Size Properties mechanism defined in
[RFC4331] that can be used for storage control. Specifically, two
key properties are defined per resource: "quota-available-bytes" and
"quota-used-bytes".
WebDAV does not define protocol operation for storage resource
control. However servers typically perform this function via
implementation algorithms in conjunction with the storage related
properties discussed above.
A.2.2. WebDAV Support for DECADE Standard Transport Protocol Primitives
SDT is used to write objects and read (download) objects from a
DECADE server. The object can be either a self-contained object such
as a multimedia file or a chunk from a P2P system.
A.2.2.1. Writing Primitives
Writing involves uploading objects to the server. WebDAV supports
PUT and POST as described in Appendix A.1.2.1. WebDAV LOCK/UNLOCK
functionality is not needed as DECADE assumes immutable data objects.
Therefore, resources cannot be edited and so do not need to be
locked. This approach should help to greatly simplify DECADE
implementations as the LOCK/UNLOCK functionality is quite complex.
A.2.2.2. Downloading Primitives
Downloading involves fetching of an object from the server. WebDAV
supports GET and HEAD as described in Appendix A.1.2.2. WebDAV LOCK/
UNLOCK functionality is not needed as DECADE assumes immutable data
objects.
A.2.3. Other Operations
WebDAV supports DELETE as described in Appendix A.1.4. In addition
WebDAV supports COPY and MOVE methods. The COPY operation creates a
duplicate of the source resource identified by the Request-URI, in
the destination resource identified by the URI in the Destination
header.
The MOVE operation on a resource is the logical equivalent of a COPY,
followed by consistency maintenance processing, followed by a delete
of the source, where all three actions are performed in a single
operation. The consistency maintenance step allows the server to
perform updates caused by the move, such as updating all URLs, other
than the Request-URI that identifies the source resource, to point to
the new destination resource.
WebDAV also supports the concept of "collections" of resources to
support joint operations on related objects (e.g. file system
directories) within a server's namespace. For example, GET and HEAD
may be done on a single resource (as in HTTP) or on a collection.
The MKCOL operation is used to create a new collection. DECADE may
find the concept of collections to be useful if there is a need to
support directory like structures in DECADE.
WebDAV servers can be interfaced from an HTML-based user interface in
a web browser. However, it is frequently desirable to be able to
switch from an HTML-based view to a presentation provided by a native
WebDAV client, directly supporting WebDAV features. The method to
perform this in a platform-neutral mechanism is specified in the
WebDAV protocol for "mounting WebDAV servers" [RFC4709]. This type
of feature may also be attractive for DECADE clients.
A.2.4. Conclusions
WebDAV has a rich array of features that can provide a good base for
DRP and SDT for DECADE. An initial analysis finds that the following
WebDAV features will be useful for DECADE:
- access control
- properties (and PROPFIND operation)
- COPY/MOVE operations
- collections
- mounting WebDAV servers
It is recommended that the following WebDAV features NOT be used for
DECADE:
- LOCK/UNLOCK
Finally, some extensions to WebDAV may still be required to meet all
DECADE requirements. For example, defining a new WebDAV "time-to-
live" property may be useful for DECADE. Further analysis is
required to fully define the potential extensions to WebDAV to meet
all DECADE requirements.
A.3. CDMI
The Cloud Data Management Interface (CDMI) specification defines a
functional interface through which applications can store and manage
data objects in a cloud storage environment. The CDMI interface for
reading/writing data is based on standard HTTP requests, with CDMI-
specific encodings using JavaScript Object Notation (JSON). CDMI is
specified by the Storage Networking Industry Association (SNIA)
[CDMI].
A.3.1. CDMI Support for DECADE Resource Protocol Primitives
DRP provides configuration of access control and resource sharing
policies on DECADE servers.
A.3.1.1. Access Control Primitives
Access control includes mechanisms for defining the access policies
for the server, and then checking the authorization of a user before
it stores or retrieves content. CDMI defines an Access Control List
(ACL) per data object, and thus supports access control (read and/or
write) at the data object granularity. An ACL contains a set of
Access Control Entries (ACEs), where each ACE specifies a principal
(i.e. user or group of users) and a set of privileges that are
granted to that principal.
CDMI requires that an HTTP authentication mechanism be available for
the server to validate the identity of a principal (client).
Specifically, CDMI requires that either HTTP Basic Authentication or
HTTP Digest Authentication be supported. CDMI recommends that HTTP
over TLS (HTTPS) is supported to encrypt the data sent over the
network.
A.3.1.2. Communication Resource Controls Primitives
Communication resources include bandwidth (upload/download) and
number of simultaneous connected clients (connections). CDMI
supports two key data attributes which provide control over the
communication resources to a client: "cdmi_max_throughput" and
"cdmi_max_latency". These attributes are defined in the metadata for
data objects and indicate the desired bandwidth or delay for
transmission of the data object from the cloud server to the client.
A.3.1.3. Storage Resource Control Primitives
Storage resources include amount of quantity and lifetime of storage.
CDMI defines metadata for individual data objects and general storage
system configuration which can be used for storage resource control.
In particular, CDMI defines the following metadata fields:
- cdmi_data_redundancy: desired number of copies to be
maintained,
- cdmi_geographic_placement: region where object is permitted to
be stored,
- cdmi_retention_period: time interval object is to be retained,
and
- cdmi_retention_autodelete: whether object should be auto
deleted after retention period.
A.3.2. CDMI Support for DECADE Standard Transport Protocol Primitives
SDT is used to write objects and read (download) objects from a
DECADE server. The object can be either a self-contained object such
as a multimedia file or a chunk from a P2P system.
A.3.2.1. Writing Primitives
Writing involves uploading objects to the server. CDMI supports
standard HTTP methods for PUT and POST as described in
Appendix A.1.2.1.
A.3.2.2. Downloading Primitives
Downloading involves fetching of an object from the server. CDMI
supports the standard HTTP GET method as described in
Appendix A.1.2.2.
A.3.3. Other Operations
CDMI supports DELETE as described in Appendix A.1.4. CDMI also
supports COPY and MOVE operations.
CDMI supports the concept of containers of data objects to support
joint operations on related objects. For example, GET may be done on
a single data object or on an entire container.
CDMI supports a global naming scheme. Every object stored within a
CDMI system will have a globally unique object string identifier
(ObjectID) assigned at creation time.
A.3.4. Conclusions
CDMI has a rich array of features that can provide a good base for
DRP and SDT for DECADE. An initial analysis finds that the following
CDMI features may be useful for DECADE:
- access control
- storage resource control
- communication resource control
- COPY/MOVE operations
- data containers
- naming scheme
Appendix B. In-Network Storage Components Mapped to DECADE Architecture
In this section we evaluate how the basic components of an in-network In this section we evaluate how the basic components of an in-network
storage system identified in Section 3 of [RFC6392] map into the storage system identified in Section 3 of [RFC6392] map into a
DECADE architecture. DECADE-compatible system.
It is important to note that complex and/or application-specific It is important to note that complex and/or application-specific
behavior is delegated to applications instead of tuning the storage behavior is delegated to applications instead of tuning the storage
system wherever possible. system wherever possible.
B.1. Data Access Interface A.1. Data Access Interface
Users can read and write objects of arbitrary size through the DECADE Users can read and write objects of arbitrary size through the
Client's Data Controller, making use of a standard data transport. Client's Data Controller, making use of a SDT.
B.2. Data Management Operations A.2. Data Management Operations
Users can move or delete previously stored objects via the DECADE Users can move or delete previously stored objects via the Client's
Client's Data Controller, making use of a standard data transport. Data Controller, making use of a SDT.
B.3. Data Search Capability A.3. Data Search Capability
Users can enumerate or search contents of DECADE servers to find Users can enumerate or search contents of Servers to find objects
objects matching desired criteria through services provided by the matching desired criteria through services provided by the Content
Content Distribution Application (e.g., buffer-map exchanges, a DHT, Distribution Application (e.g., buffer-map exchanges, a DHT, or peer-
or peer-exchange). In doing so, End-Points may consult their local exchange). In doing so, End-Points may consult their local Data
Data Index in the DECADE Client's Data Controller. Index in the Client's Data Controller.
B.4. Access Control Authorization A.4. Access Control Authorization
All methods of access control are supported: public-unrestricted, All methods of access control are supported: public-unrestricted,
public-restricted and private. Access Control Policies are generated public-restricted and private. Access Control Policies are generated
by a Content Distribution Application and provided to the DECADE by a Content Distribution Application and provided to the Client's
Client's Resource Controller. The DECADE Server is responsible for Resource Controller. The Server is responsible for implementing the
implementing the access control checks. access control checks.
B.5. Resource Control Interface A.5. Resource Control Interface
Users can manage the resources (e.g. bandwidth) on the DECADE server Users can manage the resources (e.g. bandwidth) on the DECADE server
that can be used by other Application End-Points. Resource Sharing that can be used by other Application End-Points. Resource Sharing
Policies are generated by a Content Distribution Application and Policies are generated by a Content Distribution Application and
provided to the DECADE Client's Resource Controller. The DECADE provided to the Client's Resource Controller. The Server is
Server is responsible for implementing the resource sharing policies. responsible for implementing the resource sharing policies.
B.6. Discovery Mechanism A.6. Discovery Mechanism
The particular protocol used for discovery is outside the scope of The particular protocol used for discovery is outside the scope of
this document. However, options and considerations have been this document. However, options and considerations have been
discussed in Section 5.5. discussed in Section 5.5.
B.7. Storage Mode A.7. Storage Mode
DECADE Servers provide an object-based storage mode. Immutable data Servers provide an object-based storage mode. Immutable data objects
objects may be stored at a DECADE server. Applications may consider may be stored at a Server. Applications may consider existing blocks
existing blocks as DECADE data objects, or they may adjust block as data objects, or they may adjust block sizes before storing in a
sizes before storing in a DECADE server. Server.
Authors' Addresses Authors' Addresses
Richard Alimi Richard Alimi
Google Google
Email: ralimi@google.com Email: ralimi@google.com
Y. Richard Yang
Yale University
Email: yry@cs.yale.edu
Akbar Rahman Akbar Rahman
InterDigital Communications, LLC InterDigital Communications, LLC
Email: akbar.rahman@interdigital.com Email: akbar.rahman@interdigital.com
Dirk Kutscher Dirk Kutscher
NEC NEC
Email: dirk.kutscher@neclab.eu Email: dirk.kutscher@neclab.eu
Hongqiang Liu Y. Richard Yang
Yale University Yale University
Email: hongqiang.liu@yale.edu Email: yry@cs.yale.edu
 End of changes. 201 change blocks. 
1147 lines changed or deleted 794 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/