draft-ietf-decade-arch-05.txt   draft-ietf-decade-arch-06.txt 
DECADE R. Alimi DECADE R. Alimi
Internet-Draft Google Internet-Draft Google
Intended status: Informational A. Rahman Intended status: Informational A. Rahman
Expires: September 13, 2012 InterDigital Communications, LLC Expires: December 2, 2012 InterDigital Communications, LLC
D. Kutscher D. Kutscher
NEC NEC
Y. Yang Y. Yang
Yale University Yale University
March 12, 2012 May 31, 2012
DECADE Architecture DECADE Architecture
draft-ietf-decade-arch-05 draft-ietf-decade-arch-06
Abstract Abstract
Content Distribution Applications (e.g., P2P applications) are widely Content Distribution Applications (e.g., P2P applications) are widely
used on the Internet and make up a large portion of the traffic in used on the Internet and make up a large portion of the traffic in
many networks. One technique to improve the network efficiency of many networks. One technique to improve the network efficiency of
these applications is to introduce storage capabilities within the these applications is to introduce storage capabilities within the
networks; this is the capability to be provided by a DECADE networks; this is the capability provided by a DECADE (DECoupled
(DECoupled Application Data Enroute) compliant system. This document Application Data Enroute) compatible system. This document presents
presents an architecture for DECADE, discusses the underlying an architecture, discusses the underlying principles, and identifies
principles, and identifies key functionalities required for key functionalities required for introducing a DECADE-compatible in-
introducing in-network storage for these applications. In addition, network storage system. In addition, some examples are given to
some examples are given to illustrate these concepts in an illustrate these concepts.
informative manner.
Requirements Language Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
Status of this Memo Status of this Memo
This Internet-Draft is submitted in full conformance with the This Internet-Draft is submitted in full conformance with the
skipping to change at page 2, line 4 skipping to change at page 2, line 4
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at http://datatracker.ietf.org/drafts/current/. Drafts is at http://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 13, 2012. This Internet-Draft will expire on December 2, 2012.
Copyright Notice Copyright Notice
Copyright (c) 2012 IETF Trust and the persons identified as the Copyright (c) 2012 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
carefully, as they describe your rights and restrictions with respect carefully, as they describe your rights and restrictions with respect
to this document. Code Components extracted from this document must to this document. Code Components extracted from this document must
include Simplified BSD License text as described in Section 4.e of include Simplified BSD License text as described in Section 4.e of
the Trust Legal Provisions and are provided without warranty as the Trust Legal Provisions and are provided without warranty as
described in the Simplified BSD License. described in the Simplified BSD License.
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5
2. Functional Entities . . . . . . . . . . . . . . . . . . . . . 5 2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1. DECADE Server . . . . . . . . . . . . . . . . . . . . . . 6 2.1. DECADE-compatible Client . . . . . . . . . . . . . . . . . 5
2.2. DECADE Client . . . . . . . . . . . . . . . . . . . . . . 6 2.2. DECADE-compatible Server . . . . . . . . . . . . . . . . . 5
2.3. DECADE Storage Provider . . . . . . . . . . . . . . . . . 6 2.3. Content Provider . . . . . . . . . . . . . . . . . . . . . 6
2.4. Content Provider Using DECADE . . . . . . . . . . . . . . 6 2.4. Content Consumer . . . . . . . . . . . . . . . . . . . . . 6
2.5. Content Consumer Using DECADE . . . . . . . . . . . . . . 6 2.5. Storage Provider . . . . . . . . . . . . . . . . . . . . . 6
2.6. Content Distribution Application . . . . . . . . . . . . . 7 2.6. Content Distribution Application . . . . . . . . . . . . . 6
3. Protocol Flow . . . . . . . . . . . . . . . . . . . . . . . . 7 2.7. Application End-Point . . . . . . . . . . . . . . . . . . 6
2.8. Data Object . . . . . . . . . . . . . . . . . . . . . . . 6
3. Protocol Flow . . . . . . . . . . . . . . . . . . . . . . . . 6
3.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.2. An Example . . . . . . . . . . . . . . . . . . . . . . . . 9 3.2. An Example . . . . . . . . . . . . . . . . . . . . . . . . 8
4. Architectural Principles . . . . . . . . . . . . . . . . . . . 9 4. Architectural Principles . . . . . . . . . . . . . . . . . . . 9
4.1. Decoupled Control/Metadata and Data Planes . . . . . . . . 10 4.1. Decoupled Control/Metadata and Data Planes . . . . . . . . 9
4.2. Immutable Data Objects . . . . . . . . . . . . . . . . . . 10 4.2. Immutable Data Objects . . . . . . . . . . . . . . . . . . 10
4.3. Data Object Identifiers . . . . . . . . . . . . . . . . . 11 4.3. Data Objects With Identifiers . . . . . . . . . . . . . . 10
4.4. DECADE Data Object Naming Scheme . . . . . . . . . . . . . 12 4.4. Data Object Naming Scheme . . . . . . . . . . . . . . . . 11
4.5. Explicit Control . . . . . . . . . . . . . . . . . . . . . 13 4.5. Explicit Control . . . . . . . . . . . . . . . . . . . . . 12
4.6. Resource and Data Access Control through User 4.6. Resource and Data Access Control . . . . . . . . . . . . . 13
Delegation . . . . . . . . . . . . . . . . . . . . . . . . 14 5. System Components . . . . . . . . . . . . . . . . . . . . . . 14
5. System Components . . . . . . . . . . . . . . . . . . . . . . 15 5.1. Content Distribution Application . . . . . . . . . . . . . 14
5.1. Content Distribution Application . . . . . . . . . . . . . 15 5.2. Server . . . . . . . . . . . . . . . . . . . . . . . . . . 16
5.2. DECADE Server . . . . . . . . . . . . . . . . . . . . . . 17 5.3. Data Sequencing and Naming . . . . . . . . . . . . . . . . 18
5.3. Data Sequencing and Naming . . . . . . . . . . . . . . . . 19 5.4. Token-based Authentication and Resource Control . . . . . 20
5.4. Token-based Authentication and Resource Control . . . . . 22 5.5. Discovery . . . . . . . . . . . . . . . . . . . . . . . . 21
5.5. Discovery . . . . . . . . . . . . . . . . . . . . . . . . 23 6. DECADE Protocols . . . . . . . . . . . . . . . . . . . . . . . 22
6. DECADE Protocols . . . . . . . . . . . . . . . . . . . . . . . 24 6.1. DECADE Resource Protocol (DRP) . . . . . . . . . . . . . . 22
6.1. DECADE Resource Protocol (DRP) . . . . . . . . . . . . . . 24 6.2. Standard Data Transfer (SDT) Protocol . . . . . . . . . . 25
6.2. Standard Data Transfer (SDT) . . . . . . . . . . . . . . . 27 6.3. Server-to-Server Protocols . . . . . . . . . . . . . . . . 26
6.3. Server-to-Server Protocols . . . . . . . . . . . . . . . . 29 7. Security Considerations . . . . . . . . . . . . . . . . . . . 28
7. Security Considerations . . . . . . . . . . . . . . . . . . . 31 7.1. Threat: System Denial of Service Attacks . . . . . . . . . 28
7.1. Threat: System Denial of Service Attacks . . . . . . . . . 31 7.2. Threat: Protocol Security . . . . . . . . . . . . . . . . 29
7.2. DECADE Protocol Security Threats . . . . . . . . . . . . . 32 8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 30
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 33 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 30
9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 33 10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 31
10. References . . . . . . . . . . . . . . . . . . . . . . . . . . 34 10.1. Normative References . . . . . . . . . . . . . . . . . . . 31
10.1. Normative References . . . . . . . . . . . . . . . . . . . 34 10.2. Informative References . . . . . . . . . . . . . . . . . . 31
10.2. Informative References . . . . . . . . . . . . . . . . . . 34
Appendix A. In-Network Storage Components Mapped to DECADE Appendix A. In-Network Storage Components Mapped to DECADE
Architecture . . . . . . . . . . . . . . . . . . . . 35 Architecture . . . . . . . . . . . . . . . . . . . . 31
A.1. Data Access Interface . . . . . . . . . . . . . . . . . . 35 A.1. Data Access Interface . . . . . . . . . . . . . . . . . . 32
A.2. Data Management Operations . . . . . . . . . . . . . . . . 35 A.2. Data Management Operations . . . . . . . . . . . . . . . . 32
A.3. Data Search Capability . . . . . . . . . . . . . . . . . . 35 A.3. Data Search Capability . . . . . . . . . . . . . . . . . . 32
A.4. Access Control Authorization . . . . . . . . . . . . . . . 36 A.4. Access Control Authorization . . . . . . . . . . . . . . . 32
A.5. Resource Control Interface . . . . . . . . . . . . . . . . 36 A.5. Resource Control Interface . . . . . . . . . . . . . . . . 32
A.6. Discovery Mechanism . . . . . . . . . . . . . . . . . . . 36 A.6. Discovery Mechanism . . . . . . . . . . . . . . . . . . . 32
A.7. Storage Mode . . . . . . . . . . . . . . . . . . . . . . . 36 A.7. Storage Mode . . . . . . . . . . . . . . . . . . . . . . . 32
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 33
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 36
1. Introduction 1. Introduction
Content Distribution Applications (e.g., P2P applications) are widely Content Distribution Applications, such as Peer-to-Peer (P2P)
used on the Internet today to distribute data, and they contribute a applications, are widely used on the Internet to distribute data, and
large portion of the traffic in many networks. The DECADE-compatible they contribute a large portion of the traffic in many networks. The
architecture described in this document enables such applications to DECADE-compatible architecture described in this document enables
leverage in-network storage to achieve more efficient content such applications to leverage in-network storage to achieve more
distribution. Specifically, in many subscriber networks, it can be efficient content distribution. Specifically, in many subscriber
expensive to upgrade network equipment in the "last-mile", because it networks, it can be expensive to upgrade network equipment in the
can involve replacing equipment and upgrading wiring at individual "last-mile", because it can involve replacing equipment and upgrading
homes, businesses, and devices such as DSLAMs (Digital Subscriber wiring at individual homes, businesses, and devices such as DSLAMs
Line Access Multiplexers) and CMTSs (Cable Modem Termination Systems) (Digital Subscriber Line Access Multiplexers) and CMTSs (Cable Modem
in remote locations. Therefore, it may be cheaper to upgrade the Termination Systems) in remote locations. Therefore, it may be
core infrastructure, which involves fewer components that are shared cheaper to upgrade the core infrastructure, which involves fewer
by many subscribers. See [I-D.ietf-decade-problem-statement] for a components that are shared by many subscribers. See
more complete discussion of the problem domain and general [I-D.ietf-decade-problem-statement] for a more complete discussion of
discussions of the capabilities to be provided by a DECADE-compatible the problem domain and general discussions of the capabilities to be
system. provided by a DECADE-compatible system.
This document presents an architecture for providing in-network This document presents an architecture for providing in-network
storage that can be integrated into Content Distribution storage that can be integrated into Content Distribution
Applications. The primary focus is P2P-based content distribution, Applications. The primary focus is P2P-based content distribution,
but the architecture may be useful to other applications with similar but the architecture may be useful to other applications with similar
characteristics and requirements. See [I-D.ietf-decade-reqs] for a characteristics and requirements. See [I-D.ietf-decade-reqs] for a
definition of the target applications supported by a DECADE- definition of the target applications supported by a DECADE-
compatible system. compatible system.
The approach of this document is to define the core functionalities The approach of this document is to define the core functionalities
and protocol requirements that are needed to support in-network and protocol behaviour that are needed to support in-network storage
storage in a DECADE-compatible system. The protocol themselves are in a DECADE-compatible system. The protocol themselves are not
not selected or designed. Also, if more complex functionalities are selected or designed in this document. Some illustrative examples
needed, they should be layered on top of the DECADE-compatible system are given to help the reader understand certain concepts. These
in an application specific manner. examples are purely informational and are not meant to constrain
future protocol design or selection.
Some illustrative examples are given to help the reader understand
certain concepts. These examples are purely informational and are
not meant to guide or constrain future protocol design or selection.
2. Functional Entities
This section defines the functional entities involved in a DECADE
system. Functional entities are classified as follows:
o A physical or logical component in the DECADE architecture: DECADE
Client, DECADE Server, Content Distribution Application and
Application End Point;
o Operator of a physical or logical component in the DECADE
architecture: DECADE Storage Provider; and
o Source or sink of content distributed via the DECADE architecture:
DECADE Content Provider, and DECADE Content Consumer.
2.1. DECADE Server 2. Terminology
A DECADE server stores DECADE data inside the network, and thereafter 2.1. DECADE-compatible Client
manages both the stored data and access to that data. To reinforce
that these servers are responsible for storage of raw data, this
document also refers to them as storage servers.
2.2. DECADE Client A DECADE-compatible client uploads and/or retrieves data from DECADE-
compatible servers. We simply use the term "client" if there is no
ambiguity.
A DECADE client stores and retrieves data at DECADE Servers. 2.2. DECADE-compatible Server
2.3. DECADE Storage Provider A DECADE-compatible server stores data inside the network, and
thereafter manages both the stored data and access to that data. We
simply use the term "server" if there is no ambiguity.
A DECADE storage provider deploys and/or manages DECADE storage 2.3. Content Provider
server(s) within a network. A storage provider may also own or
manage the network in which the DECADE servers are deployed, but this
is not mandatory.
A DECADE storage provider, possibly in cooperation with one or more A client which owns (i.e. uploads and manages) storage at a DECADE-
network providers, determines deployment locations for DECADE servers compatible server.
and determines the available resources for each.
2.4. Content Provider Using DECADE 2.4. Content Consumer
A content provider using DECADE accesses DECADE storage servers (by A client which has been granted permission to retrieve data from a
way of a DECADE client) to upload and manage data. Such a content DECADE-compatible server by a Content Provider.
provider can access one or more storage servers. Such a content
provider may be a single process or a distributed application (e.g.,
in a P2P scenario), and may be either fixed or mobile.
2.5. Content Consumer Using DECADE 2.5. Storage Provider
A content consumer using DECADE accesses DECADE storage servers (by A Storage Provider deploys and/or manages DECADE-compatible server(s)
way of a DECADE client). A content consumer can access one or more within a network.
DECADE storage servers. A content consumer may be a single process
or a distributed application (e.g., in a P2P scenario), and may be
either fixed or mobile. An instance of a distributed application,
such as a P2P application, may both provide content to and consume
content from DECADE storage servers.
2.6. Content Distribution Application 2.6. Content Distribution Application
A content distribution application (as a target application for A Content Distribution Application is an application (e.g., P2P)
DECADE as described in [I-D.ietf-decade-reqs]) is a distributed designed for dissemination of a large amounts of data to multiple
application designed for dissemination of a possibly-large data set consumers. Content Distribution Applications typically divide
to multiple consumers. Content Distribution Applications typically content into smaller blocks for dissemination.
divide content into smaller blocks for dissemination.
2.6.1. Application End-Point We consider Content Distribution Applications that include a DECADE-
compatible client along with other application functionality (e.g.,
P2P video streaming client).
2.7. Application End-Point
An Application End-Point is an instance of a Content Distribution An Application End-Point is an instance of a Content Distribution
Application that makes use of DECADE server(s). A particular Application. A particular Application End-Point might be a Content
Application End-Point may be a DECADE Content Provider, a DECADE Provider, a Content Consumer, or both. For example, an Application
Content Consumer, or both. For example, an Application End-Point may End-Point might be an instance of a video streaming client, or it
be an instance of a video streaming client, or it may be the source might be the source providing the video to a set of clients.
providing the video to a set of clients.
An Application End-Point need not be actively transferring data with 2.8. Data Object
other Application End-Points to interact with the DECADE storage
system. That is, an End-Point may interact with the DECADE storage
servers as an offline activity.
3. Protocol Flow A data object is the unit of data stored and retrieved from a DECADE-
compatible server. The data object is a string of raw bytes. The
server maintains metadata associated with each data object, but the
metadata is not included in the data object.
3. Protocol Flow
3.1. Overview 3.1. Overview
A DECADE-compatible system supports two logical protocols, as shown A DECADE-compatible system will support two logical protocols, as
in Figure 1. First, the DECADE Resource Protocol (DRP) is shown in Figure 1. First, the DECADE Resource Protocol (DRP) is
responsible for communication of access control and resource responsible for communication of access control and resource
scheduling policies from a DECADE Client to a DECADE Server, as well scheduling policies between a client and a server, as well as between
as between DECADE Servers. A DECADE-compatible system includes servers. A DECADE-compatible system will include exactly one DRP for
exactly one DRP for interoperability and a common format through interoperability and a common format through which these policies can
which these policies can be communicated. be communicated.
Native Application Native Application
.-------------. Protocol(s) .-------------. .-------------. Protocol(s) .-------------.
| Application | <------------------> | Application | | Application | <------------------> | Application |
| End-Point | | End-Point | | End-Point | | End-Point |
| | | | | | | |
| .--------. | | .--------. | | .--------. | | .--------. |
| | DECADE | | | | DECADE | | | | DECADE | | | | DECADE | |
| | Client | | | | Client | | | | Client | | | | Client | |
| `--------' | | `--------' | | `--------' | | `--------' |
skipping to change at page 8, line 34 skipping to change at page 7, line 43
| | | | | | | |
| | | | | | | |
v V v V v V v V
.=============. DRP .=============. .=============. DRP .=============.
| DECADE | <------------------> | DECADE | | DECADE | <------------------> | DECADE |
| Server | <------------------> | Server | | Server | <------------------> | Server |
`=============' SDT `=============' `=============' SDT `============='
Figure 1: Generic Protocol Flow Figure 1: Generic Protocol Flow
Second, a Standard Data Transport protocol (SDT) is used to transfer Second, a Standard Data Transfer (SDT) protocol will be used to
data objects to and from a DECADE Server. A DECADE-compatible system transfer data objects to and from a server. A DECADE-compatible
may support multiple SDT's. If there are multiple SDT's, a system may support multiple SDT's. If there are multiple SDT's, a
negotiation mechanism is used to determine a suitable data transfer negotiation mechanism will be used to determine a suitable SDT
protocol between a DECADE Client and DECADE Server. between the client and server.
The two protocols (DRP and SDT) may or may not be seperate on the The two protocols (DRP and SDT) will be either separate or combined
wire. For example, DRP messages may be piggy-backed within some on the wire. If they are combined, DRP messages can be piggy-backed
extension fields provided by certain data transfer protocols. In within some extension fields provided by certain SDT protocols. In
such a scenario, DRP is technically a data structure (transported by such a scenario, DRP is technically a data structure (transported by
other protocols), but it can still be considered as a logical other protocols), but it can still be considered as a logical
protocol that provides the services of configuring DECADE resource protocol that provides the services of configuring DECADE-compatible
usage. Hence, this document considers SDT and DRP as two separate, resource usage. If the protocols are physically separate on the
logical functional components for clarity. The abstract properties wire, DRP can take the form of a separate control connection open
of the SDT and DRP are oultined but the final selection of these between the a DECADE-compatible client and server. Hence, this
protocols is left to future steps. document considers SDT and DRP as two separate, logical functional
components for clarity. The abstract properties of the SDT and DRP
are outlined below but the final selection of these protocols is
outside the scope of this document.
3.2. An Example 3.2. An Example
This section provides an example of steps in a data transfer scenario This section provides an example of steps in a data transfer scenario
involving an in-network storage system. We assume that Application involving an in-network storage system. We assume that Application
End-Point B (the receiver) is requesting a data object from End-Point B (the receiver) is requesting a data object from
Application End-Point A (the sender). Let S(A) denote A's DECADE- Application End-Point A (the sender). Let S(A) denote the DECADE-
compatible storage server. There are multiple usage scenarios (by compatible storage server to which A has access. There are multiple
choice of the Content Distribution Application). For simplicity of usage scenarios (by choice of the Content Distribution Application).
introduction, we design this example to use only a single DECADE- For simplicity of introduction, we design this example to use only a
compatible Server; Section 6.3 details a case when both A and B wish single DECADE-compatible server.
to employ DECADE-compatible Servers.
When an Application End-Point wishes to use its DECADE-compatible
storage server, it provides a token to the other Application End-
Point. The token is sent using the Content Distribution
Application's native protocol.
The steps of the example are illustrated in Figure 2. First, B The steps of the example are illustrated in Figure 2. First, B
requests a data object from A using their native protocol. Next, A requests a data object from A using their native application protocol
uses the DECADE-compatible Resource Protocol (DRP) to obtain a token. (see Section 5.1.2). Next, A uses the DRP to obtain a token. There
There are multiple ways for A to obtain the token: compute locally, are multiple ways for A to obtain the token: compute locally, or
or request from its DECADE-compatible storage server, S(A). See request from its DECADE-compatible storage server, S(A). See
Section 6.1.2 for details. A then provides the token to B (again, Section 6.1.2 for details. A then provides the token to B (again,
using their native protocol). Finally, B provides the token to S(A) using their native application protocol). Finally, B provides the
via DRP, and requests and downloads the data object via a Standard token to S(A) via DRP, and requests and downloads the data object via
Data Transport (SDT). a SDT.
.----------. .----------.
2. Obtain --------> | S(A) | <------ 2. Obtain --------> | S(A) | <------
Token / `----------' \ 4. Request and Token / `----------' \ 4. Request and
(DRP) / \ Download Object (DRP) / \ Download Object
Locally / \ (DRP + SDT) Locally / \ (DRP + SDT)
or From / \ or From / \
S(A) v 1. App Request v S(A) v 1. App Request v
.-------------. <--------------------------- .-------------. .-------------. <--------------------------- .-------------.
| Application | | Application |
| End-Point A | | End-Point B | | End-Point A | | End-Point B |
`-------------' ---------------------------> `-------------' `-------------' ---------------------------> `-------------'
3. App Response (token) 3. App Response (token)
Figure 2: Download from Storage Server Figure 2: Download from Storage Server
4. Architectural Principles 4. Architectural Principles
We identify the following key principles. We identify the following key principles that will be followed in any
DECADE-compatible system:
4.1. Decoupled Control/Metadata and Data Planes 4.1. Decoupled Control/Metadata and Data Planes
A DECADE-compatible system will support multiple content distribution A DECADE-compatible system SHOULD be able to support multiple Content
applications. A complete content distribution application implements Distribution Applications. A complete Content Distribution
a set of "control plane" functions including content search, indexing Application implements a set of "control plane" functions including
and collection, access control, ad insertion, replication, request content search, indexing and collection, access control, ad
routing, and QoS scheduling. Different content distribution insertion, replication, request routing, and QoS scheduling.
applications will have unique considerations designing the control Different Content Distribution Applications will have unique
plane functions: considerations designing the control plane functions:
o Metadata Management Scheme: Traditional file systems provide a o Metadata Management Scheme: Traditional file systems provide a
standard metadata abstraction: a recursive structure of standard metadata abstraction: a recursive structure of
directories to offer namespace management; each file is an opaque directories to offer namespace management; each file is an opaque
byte stream. In content distribution, applications may use byte stream. Content Distribution Applications may use different
different metadata management schemes. For example, one metadata management schemes. For example, one application might
application may use a sequence of blocks (e.g., for file sharing), use a sequence of blocks (e.g., for file sharing), while another
while another application may use a sequence of frames (with application might use a sequence of frames (with different sizes)
different sizes) indexed by time. indexed by time.
o Resource Scheduling Algorithms: a major competitive advantage of o Resource Scheduling Algorithms: A major advantage of many
many successful P2P systems is their substantial expertise in successful P2P systems is their substantial expertise in achieving
achieving highly efficient utilization of peer and infrastructural highly efficient utilization of peer and infrastructural
resources. For instance, many streaming P2P systems have their resources. For instance, many streaming P2P systems have their
specific algorithms in constructing topologies to achieve low- specific algorithms in constructing topologies to achieve low-
latency, high-bandwidth streaming. They continue to fine-tune latency, high-bandwidth streaming. They continuously fine-tune
such algorithms. such algorithms.
Given the diversity of control plane functions, in-network storage Given the diversity of control plane functions, a DECADE-compatible
should export basic mechanisms and allow as much flexibility as system SHOULD allow as much flexibility as possible to the control
possible to the control plane to implement specific policies. This plane to implement specific policies. This conforms to the end-to-
conforms to the end-to-end systems principle and allows innovation end systems principle and allows innovation and satisfaction of
and satisfaction of specific business goals. specific performance goals.
Decoupling control plane and data plane is not new. For example, Decoupling control plane and data plane is not new. For example,
OpenFlow [OpenFlow] is an implementation of this principle for OpenFlow [OpenFlow] is an implementation of this principle for
Internet routing, where the computation of the forwarding table and Internet routing, where the computation of the forwarding table and
the application of the forwarding table are separated. Google File the application of the forwarding table are separated. Google File
System [GoogleFileSystem] applies the principle to file system System [GoogleFileSystem] applies the principle to file system
design, by utilizing the Master to handle the meta-data management, design, by utilizing the Master to handle the meta-data management,
and the Chunk Servers to handle the data plane functions (i.e., read and the Chunk servers to handle the data plane functions (i.e., read
and write of chunks of data). NFSv4.1's pNFS extension [RFC5661] and write of chunks of data). NFSv4.1's pNFS extension [RFC5661]
also implements this principle. also implements this principle.
4.2. Immutable Data Objects 4.2. Immutable Data Objects
A property of bulk contents to be broadly distributed is that they A property of bulk content to be broadly distributed is that they
typically are immutable -- once content is generated, it is typically typically are immutable -- once content is generated, it is typically
not modified. It is not common that bulk contents such as video not modified. It is not common that bulk content such as video
frames and images need to be modified after distribution. frames and images need to be modified after distribution.
Focusing on immutable data in the data plane can substantially Focusing on immutable data in the data plane can substantially
simplify the data plane design, since consistency requirements can be simplify the data plane design, since consistency requirements can be
relaxed. It also allows effective reuse of data and de-duplication relaxed. It also simplifies reuse of data and implementation of de-
of redundant content. duplication.
Depending on its specific requirements, an application may store Depending on its specific requirements, an application may store
immutable data objects in DECADE-compatible servers such that each immutable data objects in DECADE-compatible servers such that each
data object is completely self-contained (e.g., a complete, data object is completely self-contained (e.g., a complete,
independently decodable video segment). An application may also independently decodable video segment). An application may also
divide data into blocks that require application level assembly. divide data into blocks that require application level assembly.
Many content distribution applications divide bulk content into Many Content Distribution Applications divide bulk content into
blocks for two reasons: (1) multipath: different blocks may be blocks for multiple reasons, including (1) multipath: different
fetched from different sources in parallel, and (2) faster recovery blocks might be fetched from different sources in parallel; and (2)
and verification: individual blocks may be recovered and verified. faster recovery and verification: individual blocks might be
Typically, applications use a block size larger than a single packet recovered and verified. Typically, applications use a block size
in order to reduce control overhead. larger than a single packet in order to reduce control overhead.
A DECADE-compatible system is agnostic to the nature of the data A DECADE-compatible system SHOULD be agnostic to the nature of the
objects and does not specify a fixed size for them, though a protocol data objects and SHOULD NOT specify a fixed size for them. Though a
specification based on this architecture may prescribe requirements protocol specification based on this architecture MAY prescribe
on minimum and maximum sizes by compliant implementatations. requirements on minimum and maximum sizes by compliant
Applications may consider existing blocks as data objects, or they implementations. Applications may consider existing blocks as data
may adjust block sizes before storing in the DECADE-compatible objects, or they may adjust block sizes before storing in the DECADE-
server. compatible server.
Immutable content may still be deleted. Applications may support Immutable data objects can still be deleted. Applications may
modification of existing data stored at a DECADE-compatible server support modification of existing data stored at a DECADE-compatible
through a combination of storing new data objects and deleting server through a combination of storing new data objects and deleting
existing data objects. For example, a meta-data management function existing data objects. For example, a meta-data management function
of the control plane may associate a name with a sequence of of the control plane might associate a name with a sequence of
immutable blocks. If one of the blocks is modified, the meta-data immutable blocks. If one of the blocks is modified, the meta-data
management function changes the mapping of the name to a new sequence management function changes the mapping of the name to a new sequence
of immutable blocks. of immutable blocks.
Throughout this document, all data objects/blocks are referred to as Throughout this document, all data objects/blocks are assumed to be
immutable data objects/blocks. immutable.
4.3. Data Object Identifiers 4.3. Data Objects With Identifiers
Objects that are stored in a DECADE-compatible storage server can be An object that is stored in a DECADE-compatible storage server SHOULD
accessed by content consumers via a data object identifier that has be accessed by Content Consumers via a data object identifier.
been assigned within a certain application context and that is unique
within that application context.
A content consumer may be able to access more than one storage server A Content Consumer may be able to access more than one storage
within a single application context. A data object that is server. A data object that is replicated across different storage
replicated across different storage servers managed by a DECADE- servers managed by a DECADE-compatible Storage Provider MAY still be
compatible storage provider can still be accessed by a single accessed by a single identifier.
identifier.
Since data objects are immutable, it is possible to support Since data objects are immutable, it SHALL be possible to support
persistent identifiers for data objects. persistent identifiers for data objects.
Data object identifiers for data objects MUST be created by content Data object identifiers for data objects SHOULD be created by Content
providers that upload the objects to DECADE servers. Providers that upload the objects to servers.
For some applications it is important that DECADE clients and servers 4.4. Data Object Naming Scheme
are able to validate the name-object binding for a data object, i.e.,
by verifying that a received object really corresponds to the name
that was used for requesting it (or that was provided by a sender).
Data object identifiers can support name-content binding validation
by providing message digests or so-called self-certifying naming
information -- if a specific application has this requirement.
4.4. DECADE Data Object Naming Scheme The DECADE architecture is based on data object identifiers as
described above, and the assignment/derivation of the data object
identifier to a data object depends on the data object naming scheme.
The details of data naming schemes will be provided by future DECADE-
compatible protocol/naming specifications. This document describes
naming schemes on a semantic level and specific SDTs and DRPs SHOULD
use specific representations.
The DECADE architecture is based on the data object identifier In particular, for some applications it is important that clients and
properties as described in Section 4.3, but does neither specify a servers SHOULD be able to validate the name-object binding for a data
specific scheme nor specific name-object binding validation functions object, i.e., by verifying that a received object really corresponds
(for example, hash functions). Different applications will have to the name (identifier) that was used for requesting it (or that was
specific requirements regarding security (for example, cryptographic provided by a sender). Data object identifiers can support name-
strength of hash functions), performance (for example, larger static object binding validation by providing message digests or so-called
objects vs streaming) etc. self-certifying naming information -- if a specific application has
this requirement.
The details of such naming schemes will be provided by DECADE A DECADE-compatible naming scheme follows the following general
protocol/naming specifications. This documents describes the scheme
on a semantic level will but specific SDTs and DRPs may use specific
representations. The naming scheme meets the following general
requirements: requirements:
o Different name-object binding validation mechanisms are supported; o Different name-object binding validation mechanisms MAY be
supported;
o an application (DECADE content provider) can decide what mechanism o Content Distribution Applications will decide what mechanism to
to use (or to not provide name-object validation -- for example if use, or to not provide name-object validation (e.g., if
authenticity and integrity can be ascertained by alternative authenticity and integrity can by ascertained by alternative
means); means);
o simple (digest hash based) name-object binding validation is o Applications MAY be able to construct unique names (with high
supported; and
o applications are able to construct unique names (with high
probability) without requiring a registry or other forms of probability) without requiring a registry or other forms of
coordination; and coordination; and
o names are self-describing so that a receiving entity (DECADE o Names MAY be self-describing so that a receiving entity (Content
content consumer) knows what hash function (for example) to use Consumer) knows what hash function (for example) to use for
for validating name-object binding. validating name-object binding.
For most content distribution scenarios, it will be appropriate to Some Content Distribution Applications will derive the name of a data
derive the name of a data object from the hash over the data object's object from the hash over the data object, which is made possible by
content (the raw bytes), which is made possible by the fact that the fact that DECADE-compatible objects are immutable. But there
DECADE-compatible objects are immutable. But there maybe other maybe other applications such as live streaming where object/chunk
applications such as live streaming where object/chunk names are not names will not based on hashes but rather on an enumeration scheme.
based on hashes but rather on a enumeration scheme. The DECADE The naming scheme will also enable those applications to construct
naming scheme also enables those applications to construct unique unique names.
names.
In order to enable the uniqueness, flexibility and self-describing In order to enable the uniqueness, flexibility and self-describing
properties, the DECADE naming scheme provides the following name properties, the naming scheme SHOULD provide the following name
elements: elements:
o a "type" field that indicates the name-object validation function o A "type" field that indicates the name-object validation function
type (for example, "sha-256"); type (for example, "sha-256");
o cryptographic data (such as an object hash) that corresponds to o Cryptographic data (such as an object hash) that corresponds to
the type information; and the type information; and
o (optionally) additional information such as application context or The naming scheme MAY additionally provide the following name
publisher information. elements:
o Application or publisher information.
The specific format of the name (e.g., encoding, hash algorithms, The specific format of the name (e.g., encoding, hash algorithms,
etc) is out of scope of this document, and left for protocol etc) is out of scope of this document, and is left for protocol
specification. specification.
The DECADE naming scheme can be used in scenarios where a client The DECADE-compatible naming scheme SHOULD be used in scenarios where
knows the name of a data object before it is completely stored at the a client knows the name of a data object before it is completely
server. This allows for particular optimizations, such as stored at the server. This allows for particular optimizations, such
advertising data object while the data object is being stored, as advertising data object while the data object is being stored,
removing store-and-forward delays. For example, a client A may removing store-and-forward delays. For example, a client A might
simultaneously begin storing an object to a server, and advertise simultaneously begin storing an object to a server, and advertise
that the object is available to client B. If it is supported by the that the object is available to client B. If it is supported by the
server, client B may begin downloading the object before A is server, client B might begin downloading the object before A is
finished storing the object. finished storing the object.
If object names are not based on content hashes, DECADE names can If object names are not based on hashes of the data objects
also be used in scenarios, where a client knows the name of a data themselves, names can also be used in scenarios where a client knows
object before it is locally created. the name of a data object before it is locally created.
4.5. Explicit Control 4.5. Explicit Control
To support the functions of an application's control plane, To support the functions of an application's control plane,
applications must be able to know and coordinate which data is stored applications SHOULD be able to know and coordinate which data is
at particular servers. Thus, in contrast with content caches, stored at particular servers. Thus, in contrast with traditional
applications are given explicit control over the placement (selection caches, applications are given explicit control over the placement
of a DECADE-compatible server), deletion (or expiration policy), and (selection of a DECADE-compatible server), deletion (or expiration
access control for stored data. policy), and access control for stored data.
Consider deletion/expiration policy as a simple example. An Consider deletion/expiration policy as a simple example. An
application may require that a server store content for a relatively application might require that a server store data objects for a
short period of time (e.g., for live-streaming data). Another relatively short period of time (e.g., for live-streaming data).
application may need to store content for a longer duration (e.g., Another application might need to store data objects for a longer
for video-on-demand). duration (e.g., for video-on-demand).
4.6. Resource and Data Access Control through User Delegation 4.6. Resource and Data Access Control
A DECADE-compatible system provides a shared infrastructure to be A DECADE-compatible system will provide a shared infrastructure to be
used by multiple tenants of multiple content distribution used by multiple Content Consumers and Content Providers spanning
applications. Thus, it needs to provide both resource and data multiple Content Distribution Applications. Thus, it needs to
access control. provide both resource and data access control.
4.6.1. Resource Allocation 4.6.1. Resource Allocation
There are two primary interacting entities in a DECADE-compatible There are two primary interacting entities in a DECADE-compatible
system. First, Storage Providers coordinate where storage servers system. First, Storage Providers SHOULD coordinate where storage
are provisioned and their total available resources. Second, servers are provisioned and their total available resources. Second,
Applications coordinate data transfers amongst available servers and Applications will coordinate data transfers amongst available servers
between servers and end-points. A form of isolation is required to and between servers and clients. A form of isolation is required to
enable concurrently-running Applications to each explicitly manage enable concurrently-running Applications to each explicitly manage
its own content and share of resources at the available servers. its own data objects and share of resources at the available servers.
The Storage Provider delegates the management of the resources on a The Storage Provider SHOULD delegate the management of the resources
server to applications. It means applications are able to explicitly on a server to DECADE Content Providers. This means that Content
and independently manage their own shares of resources on a DECADE Providers are able to explicitly and independently manage their own
server. shares of resources on a server.
4.6.2. User Delegations 4.6.2. User Delegations
Storage providers have the ability to explicitly manage the entities Storage Providers will have the ability to explicitly manage the
allowed to utilize the resources at a DECADE-compatible server. This entities allowed to utilize the resources at a DECADE-compatible
capability is needed for reasons such as capacity-planning and legal server. This capability is needed for reasons such as capacity-
considerations in certain deployment scenarios. planning and legal considerations in certain deployment scenarios.
The server grants a share of the resources to a user. The user may
in turn share the granted resources amongst multiple applications.
The share of resources granted by a storage provider is called a User
Delegation.
As a simple example, a DECADE-compatible Server operated by an ISP
may be configured to grant each ISP Subscriber 1.5 Mbps of bandwidth.
The ISP Subscriber may in turn divide this share of resources amongst
a video streaming application and file-sharing application which are
running concurrently.
In general, a User Delegation may be granted to an end-user (e.g., an
ISP subscriber), a Content Provider, or an application. A particular
instance of an application may make use of the storage resources:
o granted to the end-user (with the end-user's permission),
o granted to the Content Provider (with the Content Provider's The server SHOULD grant a share of the resources to a Content
permission), and/or Provider or Content Consumer. The client can in turn share the
granted resources amongst its multiple applications. The share of
resources granted by a server is called a User Delegation.
o granted to the application. As a simple example, a DECADE-compatible server operated by an ISP
might be configured to grant each ISP Subscriber 1.5 Mbit/s of
bandwidth. The ISP Subscriber might in turn divide this share of
resources amongst a video streaming application and file-sharing
application which are running concurrently.
5. System Components 5. System Components
The primary focus of this document is the architectural principles The primary focus of this document is the architectural principles
and the system components that implement them. While certain system and the system components that implement them. While certain system
components might differ amongst implementations, the document details components might differ amongst implementations, the document details
the major components and their overall roles in the architecture. the major components and their overall roles in the architecture.
To keep the scope narrow, we only discuss the primary components To keep the scope narrow, we only discuss the primary components
related to protocol development. Particular deployments may require related to protocol development. Particular deployments will require
additional components (e.g., monitoring and accounting at a server), additional components (e.g., monitoring and accounting at a server),
but they are intentionally omitted from this document. but they are intentionally omitted from this document.
5.1. Content Distribution Application 5.1. Content Distribution Application
Content Distribution Applications have many functional components. Content Distribution Applications have many functional components.
For example, many P2P applications have components and algorithms to For example, many P2P applications have components and algorithms to
manage overlay topology management, rate allocation, piece selection, manage overlay topology management, rate allocation, piece selection,
etc. In this document, we focus on the components directly employed etc. In this document, we focus on the components directly employed
to support a DECADE-compatible system. to support a DECADE-compatible system.
skipping to change at page 16, line 43 skipping to change at page 15, line 43
| Resource | Data | Resource | Data
| Protocol | Transfer | Protocol | Transfer
| (DRP) | (SDT) | (DRP) | (SDT)
v V v V
Figure 3: Application Components Figure 3: Application Components
5.1.1. Data Assembly 5.1.1. Data Assembly
A DECADE-compatible system is geared towards supporting applications A DECADE-compatible system is geared towards supporting applications
that can divide distributed contents into data objects. To that can divide distributed content into data objects. To accomplish
accomplish this, applications include a component responsible for this, applications can include a component responsible for creating
creating the individual data objects before distribution and then re- the individual data objects before distribution and then re-
assembling data objects at the Content Consumer. We call this assembling data objects at the Content Consumer. We call this
component Application Data Assembly. component the Application Data Assembly.
In producing and assembling the data objects, two important In producing and assembling the data objects, two important
considerations are sequencing and naming. A DECADE-compatible system considerations are sequencing and naming. A DECADE-compatible system
assumes that applications implement this functionality themselves. assumes that applications implement this functionality themselves.
See Section 5.3 for further discussion. See Section 5.3 for further discussion.
5.1.2. Native Protocols 5.1.2. Native Application Protocols
Applications may still use existing protocols. In particular, an In addition to the DECADE-compatible DRP/SDT, applications will also
application may reuse existing protocols primarily for control/ support their existing native application protocols (e.g., P2P
signaling. However, an application may still retain its existing control and data transfer protocols).
data transfer protocols in addition to employing a data transfer
protocol with DECADE-complliant support. This is important for
applications that are designed to be highly robust (e.g., if in-
network servers are unavailable).
5.1.3. DECADE Client 5.1.3. DECADE Client
An application needs to be modified to support a DECADE-compatible An application needs to be modified to support a DECADE-compatible
system. The DECADE Client provides the local support to an system. The client provides the local support to an application, and
application. A DECADE Client need not be embedded into the can standalone, embedded into the application, or integrated in other
application. It could be implemented alone, or could be integrated entities such as network devices themselves.
in other entities such as network devices themselves.
5.1.3.1. Resource Controller 5.1.3.1. Resource Controller
Applications may have different Resource Sharing Policies and Data Applications may have different Resource Sharing Policies and Data
Access Policies to control their resource and data in DECADE- Access Policies to control their resource and data in DECADE-
compatible servers. These policies can be existing policies of compatible servers. These policies may be existing policies of
applications (e.g., tit-for-tat) or custom policies. The specific applications or custom policies. The specific implementation is
implementation is decided by the application. decided by the application.
5.1.3.2. Data Controller 5.1.3.2. Data Controller
A DECADE-compatible system decouples the control and the data A DECADE-compatible system decouples the control and the data
transfer of applications. A Data Scheduling component schedules data transfer of applications. A Data Scheduling component schedules data
transfers according to network conditions, available Servers, and/or transfers according to network conditions, available servers, and/or
available Server resources. The Data Index indicates data available available server resources. The Data Index indicates data available
at remote servers. The Data Index (or a subset of it) may be at remote servers. The Data Index (or a subset of it) can be
advertised to other Application End-Points. A common use case for advertised to other clients. A common use case for this is to
this is to provide the ability to locate data amongst a distributed provide the ability to locate data amongst distributed Application
set of Application End-Points (i.e., a data search mechanism). End-Points (i.e., a data search mechanism such as a Distributed Hash
Table).
5.2. DECADE Server 5.2. Server
A DECADE-compatible Server stores data from Application End-Points, Figure 4 illustrates the components discussed in a DECADE-compatible
and provides control and access of those data to Application End- server. A server is not necessarily a single physical machine, it
Points. A Server is not necessarily a single physical machine, it can also be implemented as a cluster of machines.
could also be implemented as a cluster of machines.
| | | |
| DECADE | Standard | DECADE | Standard
| Resource | Data | Resource | Data
| Protocol | Transfer | Protocol | Transfer
| (DRP) | (SDT) | (DRP) | (SDT)
| | | |
.= | ================= | ======================. .= | ================= | ======================.
| | v | | | v |
| | .----------------. | | | .----------------. |
skipping to change at page 18, line 35 skipping to change at page 17, line 35
| .-----------------. | User | | | .-----------------. | User | |
| | Data Store | | Delegation | | | | Data Store | | Delegation | |
| `-----------------' | Management | | | `-----------------' | Management | |
| DECADE Server `------------' | | DECADE Server `------------' |
`==============================================' `=============================================='
Figure 4: DECADE Server Components Figure 4: DECADE Server Components
5.2.1. Access Control 5.2.1. Access Control
An Application End-Point can access its own data or other Application A client SHALL be able to access its own data or other client's data
End-Point's data (provided sufficient authorization) in DECADE- (provided sufficient authorization) in DECADE-compatible servers.
compatible servers. Application End-Points may also authorize other Clients MAY also authorize other Clients to store data. If an access
End-Points to store data. If an access is authorized by an is authorized by a Client, the server SHOULD provide access. Even if
Application End-Point, the Server will provide access. a request is authorized, it MAY still fail to complete due to
insufficient resources at the server.
Even if a request is authorized, it may still fail to complete due to
insufficient resources by either the requesting Application End-
Point, the providing Application End-Point, or the Server itself.
5.2.2. Resource Scheduling 5.2.2. Resource Scheduling
Applications apply resource sharing policies or use a custom policy. Applications will apply resource sharing policies or use a custom
Servers perform resource scheduling according to the resource sharing policy. Servers perform resource scheduling according to the
policies indicated by Application End-Points as well as configured resource sharing policies indicated by Clients as well as configured
User Delegations. User Delegations.
5.2.3. Data Store 5.2.3. Data Store
Data from applications may be stored at a DECADE-compatible Server. Data from applications will be stored at a DECADE-compatible server.
Data can be deleted from storage either explicitly or automatically Data SHOULD be deleted from storage either explicitly or
(e.g., after a TTL expiration). It may be possible to perform automatically (e.g., after a TTL expiration). It SHOULD be possible
optimizations in certain cases, such as avoiding writing temporary to perform optimizations in certain cases, such as avoiding writing
data (e.g., live streaming) to persistent storage, if appropriate temporary data (e.g., live streaming) to persistent storage, if
storage hints are supported by the SDT. appropriate storage hints are supported by the SDT.
5.3. Data Sequencing and Naming 5.3. Data Sequencing and Naming
In order to provide a simple and generic interface, the DECADE- In order to provide a simple and generic interface, the DECADE-
compatible Server is only responsible for storing and retrieving compatible server will only be responsible for storing and retrieving
individual data objects. Furthermore, a DECADE-compatible system individual data objects. Furthermore, a DECADE-compatible system
uses its own simple naming scheme that provides uniqueness (with high will use its own naming scheme that provides uniqueness (with high
probability) between data objects, even across multiple applications. probability) between data objects, even across multiple applications.
5.3.1. DECADE Data Object Naming Scheme 5.3.1. Data Object Naming Scheme
The name of a data object is derived from the hash over the data
object's content (the raw bytes), which is made possible by the fact
that bjects are immutable. This scheme has multiple appealing
properties:
o Simple integrity verification
o Unique names (with high probability)
o Application independent, without a new IANA-maintained registry
The DECADE-compatible naming scheme also includes a "type" field, the
"type" identifier indicates that the name is the hash of the data
object's content and the particular hashing algorithm used. This
allows it to evolve by either changing the hashing algorithm (e.g.,
if security vulnerabilities with an existing hashing algorithm are
discovered), or moving to a different naming scheme altogether.
The specific format of the name (e.g., encoding, hash algorithms,
etc) is out of scope of this document, and left for protocol
specification.
Another advantage of this scheme is that a DECADE-compatible client
knows the name of a data object before it is completely stored at the
server. This allows for particular optimizations, such as
advertising data object while the data object is being stored,
removing store-and-forward delays. For example, a client A may
simultaneously begin storing an object to a server, and advertise
that the object is available to client B. If it is supported by the
server, client B may begin downloading the object before A is
finished storing the object.
In certain scenarios (e.g., where a client has limited computational Details of the naming scheme are discussed in Section 4.4.
power), it may be advantageous to offload the computation of a data
object's name to the Server. This possibility is not considered in
the current architecture, but may be a topic for future extensions.
5.3.2. Application Usage 5.3.2. Application Usage
Recall from Section 5.1.1 that an Application typically includes its Recall from Section 5.1.1 that an Application typically includes its
own naming and sequencing scheme. The DECADE-compatible naming own naming and sequencing scheme. The DECADE-compatible naming
format does not attempt to replace any naming or sequencing of data format SHOULD NOT attempt to replace any naming or sequencing of data
objects already performed by an Application; instead, the naming is objects already performed by an Application; instead, the naming is
intended to apply only to data objects referenced by DECADE-specific intended to apply only to data objects referenced by DECADE-specific
purposes . purposes.
DECADE-compatible names are not necessarily correlated with the
naming or sequencing used by the Application using a DECADE-
compatible client. The DECADE-compatible client is expected to
maintain a mapping from its own data objects and their names to the
DECADE-specific data objects and names. Furthermore, the DECADE-
compatible naming scheme implies no sequencing or grouping of
objects, even if this is done at the application layer.
Not only does an Application retain its own naming scheme, it may An Application using a DECADE-compatible client may use a naming and
also decide the sizes of data objects to be distributed via the sequencing scheme independent of DECADE-compatible names. The
DECADE-compatible system. This is desirable since sizes of data DECADE-compatible client SHOULD maintain a mapping from its own data
objects may impact Application performance (e.g., overhead vs. data objects and their names to the DECADE-specific data objects and
distribution delay), and the particular tradeoff is application- names.
dependent.
5.3.3. Application Usage Example 5.3.3. Application Usage Example
To illustrate these properties, this section presents multiple To illustrate these properties, this section presents multiple
examples. examples.
5.3.3.1. Application with Fixed-Size Chunks 5.3.3.1. Application with Fixed-Size Chunks
Similar to the example in Section 5.1.1, consider an Application in Similar to the example in Section 5.1.1, consider an Application in
which each individual application-layer segment of data is called a which each individual application-layer segment of data is called a
"chunk" and has a name of the form: "CONTENT_ID:SEQUENCE_NUMBER". "chunk" and has a name of the form: "CONTENT_ID:SEQUENCE_NUMBER".
Furthermore, assume that the application's native protocol uses Furthermore, assume that the application's native protocol uses
chunks of size 16KB. chunks of size 16 KiloByte (KB).
Now, assume that this application wishes to store data in DECADE- Now, assume that this application wishes to store data in DECADE-
compatible servers in data objects of size 64KB. To accomplish this, compatible servers in data objects of size 64 KB. To accomplish
it can map a sequence of 4 chunks into a single object, as shown in this, it can map a sequence of 4 chunks into a single object, as
Figure 5. shown in Figure 5.
Application Chunks Application Chunks
.---------.---------.---------.---------.---------.---------.-------- .---------.---------.---------.---------.---------.---------.--------
| | | | | | | | | | | | | |
| Chunk_0 | Chunk_1 | Chunk_2 | Chunk_3 | Chunk_4 | Chunk_5 | Chunk_6 | Chunk_0 | Chunk_1 | Chunk_2 | Chunk_3 | Chunk_4 | Chunk_5 | Chunk_6
| | | | | | | | | | | | | |
`---------`---------`---------`---------`---------`---------`-------- `---------`---------`---------`---------`---------`---------`--------
DECADE Data Objects DECADE Data Objects
.---------------------------------------.---------------------------- .---------------------------------------.----------------------------
| | | |
| Object_0 | Object_1 | Object_0 | Object_1
| | | |
`---------------------------------------`---------------------------- `---------------------------------------`----------------------------
Figure 5: Mapping Application Chunks to DECADE Data Objects Figure 5: Mapping Application Chunks to DECADE Data Objects
In this example, the Application might maintain a logical mapping In this example, the Application might maintain a logical mapping
that is able to determine the name of a DECADE-compatible data object that is able to determine the name of a DECADE-compatible data object
given the chunks contained within that data object. The name might given the chunks contained within that data object. The name might
be learned from either the original source, another endpoint with be learned from either the original source, another End-Point with
which the it is communicating, a tracker, etc. which the Application is communicating, a tracker, etc.
As long as the data contained within each sequence of chunks is As long as the data contained within each sequence of chunks is
unique, the corresponding data objects have unique names. This is globally unique, the corresponding data objects have globally unique
desired, and happens automatically if particular Application segments names. This is desired, and happens automatically if particular
the same stream of data in a different way, including different chunk Application segments the same stream of data in a different way,
sizes or different padding schemes. including different chunk sizes or different padding schemes.
5.3.3.2. Application with Continuous Streaming Data 5.3.3.2. Application with Continuous Streaming Data
Next, consider an Application whose native protocol retrieves a Consider an Application whose native protocol retrieves a continuous
continuous data stream (e.g., an MPEG2 stream) instead of downloading data stream (e.g., an MPEG2 stream) instead of downloading and
and redistributing chunks of data. Such an application could segment redistributing chunks of data. Such an application could segment the
the continuous data stream to produce either fixed-sized or variable- continuous data stream to produce either fixed-sized or variable-
sized data objects. sized data objects.
Figure 6 shows how a video streaming application might produce Figure 6 shows how a video streaming application might produce
variable-sized data objects such that each data object contains 10 variable-sized data objects such that each data object contains 10
seconds of video data. seconds of video data.
Application's Video Stream Application's Video Stream
.-------------------------------------------------------------------- .--------------------------------------------------------------------
| |
| |
skipping to change at page 22, line 26 skipping to change at page 20, line 26
DECADE Data Objects DECADE Data Objects
.--------------.--------------.--------------.--------------.-------- .--------------.--------------.--------------.--------------.--------
| | | | | | | | | |
| Object_0 | Object_1 | Object_2 | Object_3 | | Object_0 | Object_1 | Object_2 | Object_3 |
| (400 KB) | (500 KB) | (300 KB) | (300 KB) | | (400 KB) | (500 KB) | (300 KB) | (300 KB) |
`--------------`--------------`--------------`--------------`-------- `--------------`--------------`--------------`--------------`--------
Figure 6: Mapping a Continuous Data Stream to DECADE Data Objects Figure 6: Mapping a Continuous Data Stream to DECADE Data Objects
Similar to the previous example, the Application might maintain a Similar to the previous example, the Application might maintain a
mapping that is able to determine the name of a DECADE data object mapping that is able to determine the name of a data object given the
given the time offset of the video chunk. time offset of the video chunk.
5.4. Token-based Authentication and Resource Control 5.4. Token-based Authentication and Resource Control
A key feature of a DECADE-compatible system is that a Client can A key feature of a DECADE-compatible system is that a client can
authorize other Clients to store or retrieve data objects from the authorize other Clients to store or retrieve data objects from the
in-network storage. A token-based authentication scheme is used to in-network storage. A token-based authentication scheme is used to
accomplish this. accomplish this.
Specifically, an entity trusted by a Client generates a digitally- Specifically, an entity trusted by a client generates a signed token
signed token with particular properties (see Section 6.1.2 for with particular properties (see Section 6.1.2 for details). The
details). The Client distributes this token to other Clients which client then distributes this token to other Clients which then use
then use the token when sending requests to the DECADE-compatible the token when sending requests to the DECADE-compatible server.
Server. Upon receiving a token, the Server validates the signature Upon receiving a token, the server validates the signature and the
and the operation being performed. operation being performed (e.g. PUT, GET).
This is a simple scheme, but has some important advantages over an This is a simple scheme, but has some important advantages over an
alternate approach in which a Client explicitly manipulates an Access alternate approach in which a client explicitly manipulates an Access
Control List (ACL) associated with each data object. In particular, Control List (ACL) associated with each data object. In particular,
it has the following advantages when applied to DECADE-compliant it has the following advantages when applied to DECADE-compatible
target applications: target applications:
o Authorization policies are implemented within the Application; an o Authorization policies are implemented within the Application; an
Application explicitly controls when tokens are generated and to Application explicitly controls when tokens are generated and to
whom they are distributed. whom they are distributed.
o Fine-grained access and resource control can be applied to data o Fine-grained access and resource control can be applied to data
objects; see Section 6.1.2 for the list of restrictions that can objects; see Section 6.1.2 for the list of restrictions that can
be enforced with a token. be enforced with a token.
o There is no messaging between a Client and Server to manipulate o There is no messaging between a client and server to manipulate
data object permissions. This can simplify, in particular, data object permissions. This can simplify, in particular,
Applications which share data objects with many dynamic peers and Applications which share data objects with many dynamic peers and
need to frequently adjust access control policies attached to data need to frequently adjust access control policies attached to data
objects. objects.
o Tokens can provide anonymous access, in which a Server does not o Tokens can provide anonymous access, in which a server does not
need to know the identity of each Client that accesses it. This need to know the identity of each client that accesses it. This
enables a Client to send tokens to Clients in other administrative enables a client to send tokens to clients in other administrative
or security domains, and allow them to read or write data objects or security domains, and allow them to read or write data objects
from its storage. from its storage.
In addition to Clients applying access control policies to data In addition to clients applying access control policies to data
objects, the Server may be configured to apply additional policies objects, the server MAY be configured to apply additional policies
based on user, object, geographic location, etc. A Client may thus based on user, object, geographic location, etc. A client might thus
be denied access even though it possess a valid token. be denied access even though it possess a valid token.
Existing protocols (e.g., OAuth [RFC5849]) implement similar referral Existing protocols (e.g., OAuth [RFC5849]) implement similar referral
mechanisms using tokens. A protocol specification of this mechanisms using tokens. A protocol specification of this
architecture will prefer to use existing mechanisms wherever architecture SHOULD endeavor to use existing mechanisms wherever
possible. possible.
5.5. Discovery 5.5. Discovery
A DECADE-compatible systme includes a discovery mechanism through A DECADE-compatible system SHOULD include a discovery mechanism
which clients locate an appropriate Server. [I-D.ietf-decade-reqs] through which clients locate an appropriate server.
details specific requirements of the discovery mechanism; this [I-D.ietf-decade-reqs] details specific requirements of the discovery
section discusses how they relate to other principles outlined in mechanism; this section discusses how they relate to other principles
this document. outlined in this document.
A discovery mechanism allows a client to determine an IP address or A discovery mechanism SHOULD allow a client to determine an IP
some other identifier that can be resolved to locate the server for address or some other identifier that can be resolved to locate the
which the client will be authorized to generate tokens (via DRP). server for which the client will be authorized to generate tokens
(The discovery mechanism may also result in an error if no such (via DRP). (The discovery mechanism might also result in an error if
servers can be located.) After discovering one or more servers, a no such servers can be located.) After discovering one or more
client may distribute load and requests across them (subject to servers, a client can distribute load and requests across them
resource limitations and policies of the servers themselves) (subject to resource limitations and policies of the servers
according to the policies of the Application End-Point in which it is themselves) according to the policies of the Application End-Point in
embedded. which it is embedded.
The particular protocol used for discovery is out of scope of this The particular protocol used for discovery is out of scope of this
document, but any specification will re-use standard protocols document, but any specification SHOULD re-use standard protocols
wherever possible. wherever possible.
The discovery mechanism outlined here does not provide the ability to The discovery mechanism outlined here does not provide the ability to
locate arbitrary DECADE-compatible servers to which a client might locate arbitrary DECADE-compatible servers to which a client might
obtain tokens from others. To do so requires application-level obtain tokens from others. To do so requires application-level
knowledge, and it is assumed that this functionality is implemented knowledge, and it is assumed that this functionality is implemented
in the Content Distribution Application, or if desired and needed, as in the Content Distribution Application.
an extension to a DECADE-compatible system.
6. DECADE Protocols 6. DECADE Protocols
This section presents the DECADE Resource Protocol (DRP) and the This section presents the DRP and the SDT protocol in terms of
Standard Data Transfer (SDT) in terms of abstract protocol abstract protocol interactions that are intended to be mapped to
interactions that are intended to mapped to specific protocols. In specific protocols. In general, the DRP/SDT functionality between a
general, the DRP/SDT functionality between a DECADE-compatible DECADE-compatible client-server are very similar to the DRP/SDT
client-server are very similiar to the DRP/SDT functionality between functionality between running between server-server. Any differences
running between server-server. Any differences are highlighted are highlighted below.
below.
The DRP is the protocol used by a DECADE-compatible client to The DRP will be the protocol used by a DECADE-compatible client to
configure the resources and authorization used to satisfy requests configure the resources and authorization used to satisfy requests
(reading, writing, and management operations concerning objects) at a (reading, writing, and management operations concerning objects) at a
server. The SDT is used to send the operations to the server. server. The SDT will be used to send the data to the server.
Necessary DRP metadata is supplied using mechanisms in the SDT that
are provided for extensibility (e.g., additional request parameters
or extension headers).
6.1. DECADE Resource Protocol (DRP) 6.1. DECADE Resource Protocol (DRP)
DRP provides configuration of access control and resource sharing DRP will provide configuration of access control and resource sharing
policies on DECADE-compatible servers. A content distribution policies on DECADE-compatible servers. A Content Distribution
application, e.g., a live P2P streaming session, may have permission Application, e.g., a live P2P streaming session, can have permission
to manage data at several servers, for instance, servers in different to manage data at several servers, for instance, servers in different
operator domains, and DRP allows one instance of such an application, operator domains, and DRP allows one instance of such an application,
e.g., an application endpoint, to apply access control and resource e.g., an Application End-Point, to apply access control and resource
sharing policies on each of them. sharing policies on each of them.
6.1.1. Controlled Resources 6.1.1. Controlled Resources
On a single DECADE-compatible server, the following resources may be On a single DECADE-compatible server, the following resources SHOULD
managed: be managed:
communication resources: Servers have limited communication o Communication resources in terms of bandwidth (upload/download)
resources in terms of bandwidth (upload/download) but also in and also in terms of number of connected clients (connections) at
terms of number of connected clients (connections) at a time. a time.
storage resources: Servers have limited storage resources. o Storage resources.
6.1.2. Access and Resource Control Token 6.1.2. Access and Resource Control Token
A token includes the following information: A token SHOULD include the following information:
Permitted operations (e.g., read, write) o Permitted operations (e.g., read, write)
Permitted objects (e.g., names of data objects that may be read or o Permitted objects (e.g., names of data objects that might be read
written) or written)
Expiration time o Expiration time
Priority for bandwidth given to requested operation (e.g., a o Priority for bandwidth given to requested operation (e.g., a
weight used in a weighted bandwidth sharing scheme) weight used in a weighted bandwidth sharing scheme)
Amount of data that may be read or written o Amount of data that might be read or written
The particular format for the token is out of scope of this document. The tokens SHOULD be generated by an entity trusted by both the
DECADE-compatible client and server at the request of a DECADE-
compatible client. For example this entity could be the client, a
server trusted by the client, or another server managed by a Storage
Provider trusted by the client. It is important for a server to
trust the entity generating the tokens since each token may incur a
resource cost on the server when used. Likewise, it is important for
a client to trust the entity generating the tokens since the tokens
grant access to the data stored at the server.
The tokens are generated by a trusted entity at the request of a Upon generating a token, a client MAY distribute it to another client
DECADE-compatible Client. It is out of scope of this document to (e.g., via their native application protocol). The receiving client
identify which entity serves this purpose, but examples include the MAY then connect to the sending client's server and perform any
Client itself, a Server trusted by the Client, or another server operation permitted by the token. The token SHOULD be sent along
managed by a Storage Provider trusted by the Client. with the operation. The server SHOULD validate the token to identify
the client that issued it and whether the requested operation is
permitted by the contents of the token. If the token is successfully
validated, the server SHOULD apply the resource control policies
indicated in the token while performing the operation.
Upon generating a token, a Client may distribute it to another Client Tokens SHOULD include a unique identifier to allow a server to detect
(e.g., via their native Application protocol). The receiving Client when a token is used multiple times and reject the additional usage
may then connect to the sending Client's Server and perform any attempts. Since usage of a token incurs resource costs to a server
operation permitted by the token. The token must be sent along with (e.g., bandwidth and storage) and a Content Provider may have a
the operation. The Server validates the token to identify the Client limited budget (see Section 4.6), the a Content Provider should be
that issued it and whether the requested operation is permitted by able to indicate if a token may be used multiple times.
the contents of the token. If the token is successfully validated,
the Server applies the resource control policies indicated in the
token while performing the operation.
Tokens include a unique identifier to allow a Server to detect when a It SHOULD be possible for DRP to allow tokens to apply to a batch of
token is used multiple times. operations to reduce communication overhead required between clients.
A request sent in this way explicitly denotes the objects to which it
applies.
It is possible for DRP to allow tokens to apply to a batch of It SHOULD be possible to revoke tokens after they are generated.
operations to reduce communication overhead required between Clients. This could be accomplished by supplying the server the unique
identifiers of the tokens which are to be revoked.
6.1.3. Status Information 6.1.3. Status Information
DRP provides a request service for status information that clients DRP SHOULD provide a request service for status information that
can use to request information from a server. clients can use to request information from a server.
status information per application context on a specific server: Status information on a specific server: Access to such status
Access to such status information requires client authorization, information SHOULD require client authorization, i.e., clients
i.e., clients need to be authorized to access status information need to be authorized to access the requested status information.
for a specific application context. This authorization (and the This authorization is based on the user delegation concept as
mapping to application contexts) is based on the user delegation described in Section 4.6. The following status information
concept as described in Section 4.6. The following status elements SHOULD be obtained:
information elements can be obtained:
* list of associated objects (with properties) * List of associated objects (with properties)
* resources used/available * Resources used/available
* list of servers to which objects have been distributed (in a The following information elements MAY additionally be available:
* List of servers to which objects have been distributed (in a
certain time-frame) certain time-frame)
* list of clients to which objects have been distributed (in a * List of clients to which objects have been distributed (in a
certain time-frame) certain time-frame)
For the list of servers/clients to which objects have been For the list of servers/clients to which objects have been
distributed to, the server can decide on time bounds for which distributed to, the server SHOULD be able to decide on time bounds
this information is stored and specify the corresponding time for which this information is stored and specify the corresponding
frame in the response to such requests. Some of this information time frame in the response to such requests. Some of this
can be used for accounting purposes, e.g., the list of clients to information may be used for accounting purposes, e.g., the list of
which objects have been distributed. clients to which objects have been distributed.
access information per application context on a specific server: Access information on a specific server: Access information MAY be
Access information can be provided for accounting purposes, for provided for accounting purposes, for example, when Content
example, when application service providers are interested to Providers are interested in access statistics for resources and/or
maintain access statistics for resources and/or to perform to perform accounting per user. Again, access to such information
accounting per user. Again, access to such information requires requires client authorization SHOULD based on the delegation
client authorization based on the user delegation concept as concept as described in Section 4.6. The following type of access
described in Section 4.6. The following access information information elements MAY be requested:
elements can be requested:
* what objects have been accessed how many times * What objects have been accessed how many times
* access tokens that a server as seen for a given object * Access tokens that a server as seen for a given object
The server can decide on time bounds for which this information is The server SHOULD decide on time bounds for which this information
stored and specify the corresponding time frame in the response to is stored and specify the corresponding time frame in the response
such requests. to such requests.
6.1.4. Object Attributes 6.1.4. Object Attributes
Objects that are stored on a DECADE-compatible server may have Objects that are stored on a DECADE-compatible server SHOULD have
associated attributes (in addition to the object identifier and the associated attributes (in addition to the object identifier and data
actual content) that relate to the data storage and its management. object) that relate to the data storage and its management. These
These attributes may be used by the server (and possibly the attributes may be used by the server (and possibly the underlying
underlying storage system) to perform specialized processing or storage system) to perform specialized processing or handling for the
handling for the data object, or to attach related server or storage- data object, or to attach related server or storage-layer properties
layer properties to the data object. These attributes have a scope to the data object. These attributes have a scope local to a server.
local to a server. In particular, these attributes are not applied In particular, these attributes SHOULD NOT be applied to a server or
to a server or client to which a data object is copied. client to which a data object is copied.
Depending on authorization, clients may get or set such attributes.
This authorization (and the mapping to application contexts) is based
on the user delegation concept as described in Section 4.6. The
architecture does not limit the set of permissible attributes, but
rather specifies a set of baseline attributes that should be
supported by implementations.
Suggested attributes are the following: Depending on authorization, clients SHOULD be permitted to get or set
such attributes. This authorization is based on the delegation
concept as described in Section 4.6. The architecture does not limit
the set of permissible attributes, but rather specifies a set of
baseline attributes that SHOULD be supported:
Expiration Time: Time at which the object may be deleted Expiration Time: Time at which the object might be deleted
object size: in bytes Object size: In bytes
Media type Media type Labelling of type as per [RFC4288]
access statistics: how often the object has been accessed (and what Access statistics: How often the object has been accessed (and what
tokens have been used) tokens have been used)
The Object Attributes defined here are distinct from application The Object Attributes defined here are distinct from application
metadata (see Section 4.1). Application metadata is custom metadata (see Section 4.1). Application metadata is custom
information that an application may wish to associate with a data information that an application might wish to associate with a data
object to understand its semantic meaning (e.g., whether it is video object to understand its semantic meaning (e.g., whether it is video
and/or audio, its playback length in time, or its index in a stream). and/or audio, its playback length in time, or its index in a stream).
If an application wishes to store such metadata persistently, it can If an application wishes to store such metadata persistently, it can
be stored within data objects themselves. be stored within data objects themselves.
6.2. Standard Data Transfer (SDT) 6.2. Standard Data Transfer (SDT) Protocol
A DECADE-compatible server provide a data access interface, and SDT A DECADE-compatible server will provide a data access interface, and
is used to write objects to a server and to read (download) objects the SDT will be used to write objects to a server and to read
from a server. Semantically, SDT is a client-server protocol, i.e., (download) objects from a server. Semantically, SDT is a client-
the server always responds to client requests. server protocol, i.e., the server always responds to client requests.
6.2.1. Writing/Uploading Objects 6.2.1. Writing/Uploading Objects
To write an object, a client first generates the object's name (see To write an object, a client first generates the object's name (see
Section 5.3), and then uploads the object to a server and supplies Section 5.3), and then uploads the object to a server and supplies
the generated name. The name can be used to access (download) the the generated name. The name can be used to access (download) the
object later, e.g., the client can pass the name as a reference to object later, e.g., the client can pass the name as a reference to
other client that can then refer to the object. other client that can then refer to the object.
Objects can be self-contained objects such as multimedia resources, Objects can be self-contained objects such as multimedia resources,
files etc., but also chunks, such as chunks of a P2P distribution files etc., but also chunks, such as chunks of a P2P distribution
protocol that can be part of a containing object or a stream. protocol that can be part of a containing object or a stream.
A server may accept download requests for an object that is still If supported, a server can accept download requests for an object
being uploaded. that is still being uploaded.
The application that originates the objects generates DECADE- The application that originates the objects generates DECADE-
compatible object names according to the naming specification in compatible object names according to the naming specification in
Section 5.3. The naming scheme provides that the name is unique. Section 5.3. Clients (as parts of application entities) upload a
Clients (as parts of application entities) upload a named object to a named object to a server. If supported, a server can verify the
server. A server may verify the integrity and other security integrity and other security properties of uploaded objects.
properties of uploaded objects.
In the following we provide an abstract specification of the upload
operation that we name 'PUT METHOD'.
Method PUT:
Parameters:
NAME: The naming of the object according to Section 5.3
OBJECT: The object itself.
Description: The PUT method is used by a DECADE-compatible client to
upload an object with an associated name 'NAME' to a DECADE-
compatible server.
RESPONSES: The server responds with one the following response
messages:
CREATED: The object has been uploaded successfully and is now
available under the specified name.
ERRORs:
VALIDATION_FAILED: The contents of the data object received by
the server did not match the provided name (i.e., hash
validation failed).
PERMISSION_DENIED: The client lacked sufficient permission to
store the object.
Specifics regarding error handling, including additional error
conditions, precedence for returned errors and its relation
with server policy, are deferred to eventual protocol
specification.
6.2.2. Downloading Objects 6.2.2. Downloading Objects
A client can request named objects from a server. In the following, A client can request named objects from a server. In a corresponding
we provide an abstract specification of the download operation that request message, a client specifies the object name and a suitable
we name 'GET METHOD'. access and resource control token. The server checks the validity of
the received token and its associated resource usage-related
Method GET: properties.
Parameters:
NAME: The naming of the object according to Section 5.3.
Description: The GET method is used by a client to download an
object with an associated name 'NAME' from a server.
RESPONSES: The server responds with one the following response
messages:
OK: The request has succeeded, and an entity corresponding to the
requested resource is sent in the response.
ERRORs:
NOT_FOUND: The server has not found anything matching the
request object name.
PERMISSION_DENIED: The client lacked sufficient permission to If the named object exists on the server and then token has been
read the object. validated, the server delivers the requested object in a response
message.
NOT_AVAILABLE: The data object exists but is currently If the object cannot be delivered the server provides an
unavailable for download (e.g., due to server policy). corresponding status/reason information in a response message.
Specifics regarding error handling, including additional error Specifics regarding error handling, including additional error
conditions (e.g. overload), precedence for returned errors and conditions (e.g. overload), precedence for returned errors and its
its relation with server policy, are deferred to eventual relation with server policy, are deferred to eventual protocol
protocol specification. specification.
6.3. Server-to-Server Protocols 6.3. Server-to-Server Protocols
An important feature of a DECADE-compatible system is the capability An important feature of a DECADE-compatible system is the capability
for one server to directly download objects from another server. for one server to directly download objects from another server.
This capability allows Applications to directly replicate data This capability allows Applications to directly replicate data
objects between servers without requiring end-hosts to use uplink objects between servers without requiring end-hosts to use uplink
capacity to upload data objects to a different server. capacity to upload data objects to a different server.
DRP and SDT support operations directly between servers. Servers are DRP and SDT will support operations directly between servers.
not assumed to trust each other nor are configured to do so. All Servers are not assumed to trust each other nor are configured to do
data operations are performed on behalf of clients via explicit so. All data operations are performed on behalf of clients via
instruction. However, the objects being processed do not necessarily explicit instruction. However, the objects being processed do not
have to originate or terminate at the client (i.e. the object may be necessarily have to originate or terminate at the client (i.e. the
limited to being exchanged between servers even if the instruction is object might be limited to being exchanged between servers even if
triggered by the client). clients thus must be able to indicate to a the instruction is triggered by the client). Clients thus will be
server the following additional parameters: able to indicate to a server the following additional parameters:
o which remote server(s) to access; o Which remote server(s) to access;
o the operation to be performed (e.g. PUT, GET); and o The operation to be performed (e.g. PUT, GET); and
o The Content Provider at the remote server from which to retrieve
the object (for a GET), or in which the object is to be stored
(for a PUT).
o Credentials indicating permission to perform the operation at the o Credentials indicating permission to perform the operation at the
remote server. remote server.
In this way, a server acts as a proxy for a client, and a client may
instantiate requests via that proxy. The operations are performed as
if the original requester had its own client co-located with the
server. It is this mode of operation that provides substantial
savings in uplink capacity. This mode of operation may also be
triggered by an administrative/management application outside the
architecture.
6.3.1. Operational Overview
Server-to-server support is focused on reading and writing data Server-to-server support is focused on reading and writing data
objects between servers. A DECADE-compatible GET or PUT request may objects between servers. The data object referred to at the remote
supply the following additional parameters: server is the same as the original request. Object attributes (see
Section 6.1.4) might also be specified in the request to the remote
REMOTE_SERVER: Address of the remote server. The format of the server.
address is out-of-scope of this document.
REMOTE_USER: The account at the remote server from which to retrieve
the object (for a GET), or in which the object is to be stored
(for a PUT).
TOKEN: Credentials to be used at the remote server.
These parameters are used by the server to instantiate a request to In this way, a server acts as a proxy for a client, and a client can
the specified remote server. It is assumed that the data object instantiate requests via that proxy. The operations will be
referred to at the remote server is the same as the original request. performed as if the original requester had its own client co-located
Object attributes (see Section 6.1.4) may also be specified in the with the server. It is this mode of operation that provides
request to the remote server. substantial savings in uplink capacity. This mode of operation can
also be triggered by an administrative/management application outside
the architecture.
When a client sends a request to a server with these additional When a client sends a request to a server with these additional
parameters, it is giving the server permission to act (proxy) on its parameters, it is giving the server permission to act (proxy) on its
behalf. Thus, it would be prudent for the supplied token to have behalf. Thus, it would be prudent for the supplied token to have
narrow privileges (e.g., limited to only the necessary data objects) narrow privileges (e.g., limited to only the necessary data objects)
or validity time (e.g., a small expiration time). or validity time (e.g., a small expiration time).
In the case of a GET operation, the server is to retrieve the data In the case of a GET operation, the server is to retrieve the data
object from the remote server using the specified credentials (via a object from the remote server using the specified credentials (via a
GET request to the remote server), and then optionally return the GET request to the remote server), and then optionally return the
object to a client. In the case of a PUT operation, the server is to object to a client. In the case of a PUT operation, the server is to
store the object to the remote server using the specified credentials store the object to the remote server using the specified credentials
(via a PUT request to the remote server). The object may optionally (via a PUT request to the remote server). The object might
be uploaded from the client or may already exist at the proxying optionally be uploaded from the client or might already exist at the
server. proxy server.
7. Security Considerations 7. Security Considerations
In general, the security considerations mentioned in In general, the security considerations mentioned in
[I-D.ietf-decade-problem-statement] apply to this document as well. [I-D.ietf-decade-problem-statement] apply to this document as well.
A DECADE-compatible system provides a distributed storage service for A DECADE-compatible system provides a distributed storage service for
content distribution and similar applications. The system consists content distribution and similar applications. The system consists
of servers and clients that use these servers to upload data objects, of servers and clients that use these servers to upload data objects,
to request distribution of data objects, and to download data to request distribution of data objects, and to download data
objects. Such a system is employed in an overall application context objects. Such a system is employed in an overall application context
-- for example in a P2P content distribution application, and it is -- for example in a P2P Content Distribution Application, and it is
generally expected that DECADE-compatible clients take part in expected that DECADE-compatible clients take part in application-
application-specific communication sessions. specific communication sessions.
The security considerations here focus on threats related to the The security considerations here focus on threats related to the
DECADE-compatible system and its communication services, i.e., the DECADE-compatible system and its communication services, i.e., the
DRP/SDT protocols that have been described in an abstract fashion in DRP/SDT protocols that have been described in an abstract fashion in
this document. this document.
7.1. Threat: System Denial of Service Attacks 7.1. Threat: System Denial of Service Attacks
A DECADE-compatible network of servers may be used to distribute data A DECADE-compatible network of servers might be used to distribute
objects from one client to a set of servers using the server-to- data objects from one client to a set of servers using the server-to-
server communication feature that a client can request when uploading server communication feature that a client can request when uploading
an object. Multiple clients uploading many objects at different an object. Multiple clients uploading many objects at different
servers at the same time and requesting server-to-server distribution servers at the same time and requesting server-to-server distribution
for them could thus mount massive distributed denial of service for them could thus mount massive distributed denial of service
(DDOS) attacks, overloading a network of servers. (DDOS) attacks, overloading a network of servers.
This threat is addressed by its access control and resource control This threat is addressed by its access control and resource control
framework. Servers can require application endpoints to be framework. Servers can require Application End-Points to be
authorized to store and to download objects, and application authorized to store and to download objects, and Application End-
endpoints can delegate authorization to other application endpoints Points can delegate authorization to other Application End-Points
using the token mechanism. using the token mechanism.
Of course the effective security of this approach depends on the Of course the effective security of this approach depends on the
strength of the token mechanism. See below for a discussion of this strength of the token mechanism. See below for a discussion of this
and related communication security threats. and related communication security threats.
Denial of Service Attacks against a single server (directing many Denial of Service Attacks against a single server (directing many
requests to that server) may still lead to considerable load for requests to that server) might still lead to considerable load for
processing requests and invalidating tokens. A SDT therefore MUST processing requests and invalidating tokens. A SDT therefore MUST
provide a redirection mechanism as described as a requirement in provide a redirection mechanism as described as a requirement in
[I-D.ietf-decade-reqs]. [I-D.ietf-decade-reqs].
7.2. DECADE Protocol Security Threats 7.2. Threat: Protocol Security
7.2.1. Threat: Authorization Mechanisms Compromised 7.2.1. Threat: Authorization Mechanisms Compromised
A DECADE-compatible system does not require application endpoints to A DECADE-compatible system does not require Application End-Points to
authenticate in order to access a server for downloading objects, authenticate in order to access a server for downloading objects,
since authorization is not based on endpoint or user identities but since authorization is not based on End-Point or user identities but
on the delegation-based authorization mechanism. Hence, most on the delegation-based authorization mechanism. Hence, most
protocol security threats are related to the authorization scheme. protocol security threats are related to the authorization scheme.
The security of the token mechanism depends on the strength of the The security of the token mechanism depends on the strength of the
token mechanism and on the secrecy of the tokens. A token can token mechanism and on the secrecy of the tokens. A token can
represent authorization to store a certain amount of data, to represent authorization to store a certain amount of data, to
download certain objects, to download a certain amount of data per download certain objects, to download a certain amount of data per
time etc. If it is possible for an attacker to guess, construct or time etc. If it is possible for an attacker to guess, construct or
simply obtain tokens, the integrity of the data maintained by the simply obtain tokens, the integrity of the data maintained by the
servers, or at least the affected application context on servers, is servers is compromised.
compromised.
This is a general security threat that applies to authorization This is a general security threat that applies to authorization
delegation schemes. Specifications of existing delegation schemes delegation schemes. Specifications of existing delegation schemes
such as OAuth [RFC5849] discuss these general threats in detail. We such as OAuth [RFC5849] discuss these general threats in detail. We
can say that the DRP has to specify appropriate algorithms for token can say that the DRP has to specify appropriate algorithms for token
generation. Moreover, authorization tokens should have a limited generation. Moreover, authorization tokens should have a limited
validity period that should be specified by the application. Token validity period that should be specified by the application. Token
confidentiality should be provided by application protocols that confidentiality should be provided by application protocols that
carry tokens, and the SDT and DRP should provide secure carry tokens, and the SDT and DRP should provide secure
(confidential) communication modes. (confidential) communication modes.
7.2.2. Threat: Object Spoofing 7.2.2. Threat: Object Spoofing
In a DECADE-compliant system, an application endpoint is referring In a DECADE-compatible system, an Application End-Point is referring
other application endpoints to servers to download a specified data other Application End-Points to servers to download a specified data
objects. An attacker could "inject" a faked version of the object objects. An attacker could "inject" a faked version of the object
into this process, so that the downloading endpoint effectively into this process, so that the downloading End-Point effectively
receives a different object (compared to what the uploading endpoint receives a different object (compared to what the uploading End-Point
provided). As result, the downloading endpoint believes that is has provided). As result, the downloading End-Point believes that is has
received an object that corresponds to the name it was provided received an object that corresponds to the name it was provided
earlier, whereas in fact it is a faked object. Corresponding attacks earlier, whereas in fact it is a faked object. Corresponding attacks
could be mounted against the application protocol (that is used for could be mounted against the application protocol (that is used for
referring other endpoints to servers), servers themselves (and their referring other End-Points to servers), servers themselves (and their
storage sub-systems), and the SDT by which the object is uploaded, storage sub-systems), and the SDT by which the object is uploaded,
distributed and downloaded. distributed and downloaded.
A DECADE-compliant systems fundamental mechanism against object A DECADE-compatible systems fundamental mechanism against object
spoofing is name-content binding validation, i.e., the ability of a spoofing is name-object binding validation, i.e., the ability of a
receiver to check whether the name he was provided and that he used receiver to check whether the name he was provided and that he used
to request an object, actually corresponds to the bits he received. to request an object, actually corresponds to the bits he received.
As described above, this allows for different forms of name-content As described above, this allows for different forms of name-object
binding, for example using content hashes, with different hash binding, for example using hashes of data objects, with different
functions (different algorithms, different digest lengths). For hash functions (different algorithms, different digest lengths). For
those application scenarios where content hashes are not applicable those application scenarios where hashes of data objects are not
(for example live-streaming) other forms of name-content binding can applicable (for example live-streaming) other forms of name-object
be used (see Section 5.3. This flexibility also addresses binding can be used (see Section 5.3). This flexibility also
cryptographic algorithm evolvability: hash functions may get addresses cryptographic algorithm evolvability: hash functions might
deprecated, better alternatives may be invented etc., so that get deprecated, better alternatives might be invented etc., so that
applications can choose appropriate mechanisms meeting their security applications can choose appropriate mechanisms meeting their security
requirements. requirements.
DECADE-compliant servers MAY perform name-content binding validation DECADE-compatible servers MAY perform name-object binding validation
on stored objects, but application endpoints MUST NOT rely on that. on stored objects, but Application End-Points MUST NOT rely on that.
In other forms: application endpoints SHOULD perform name-content In other forms: Application End-Points SHOULD perform name-object
binding validation on received objects. binding validation on received objects.
8. IANA Considerations 8. IANA Considerations
This document does not have any IANA considerations. This document does not have any IANA considerations.
9. Acknowledgments 9. Acknowledgments
We thank the following people for their contributions to this We thank the following people for their contributions to this
document: document:
skipping to change at page 34, line 4 skipping to change at page 30, line 43
David McDysan David McDysan
Borje Ohlman Borje Ohlman
Haibin Song Haibin Song
Martin Stiemerling Martin Stiemerling
Richard Woundy Richard Woundy
Ning Zong Ning Zong
10. References 10. References
10.1. Normative References 10.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997. Requirement Levels", BCP 14, RFC 2119, March 1997.
10.2. Informative References 10.2. Informative References
[RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H.,
Masinter, L., Leach, P., and T. Berners-Lee, "Hypertext
Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.
[RFC3744] Clemm, G., Reschke, J., Sedlar, E., and J. Whitehead, "Web
Distributed Authoring and Versioning (WebDAV)
Access Control Protocol", RFC 3744, May 2004.
[RFC4288] Freed, N. and J. Klensin, "Media Type Specifications and [RFC4288] Freed, N. and J. Klensin, "Media Type Specifications and
Registration Procedures", BCP 13, RFC 4288, December 2005. Registration Procedures", BCP 13, RFC 4288, December 2005.
[RFC4331] Korver, B. and L. Dusseault, "Quota and Size Properties fo
r Distributed Authoring and Versioning (DAV) Collections",
RFC 4331, February 2006.
[RFC4709] Reschke, J., "Mounting Web Distributed Authoring and
Versioning (WebDAV) Servers", RFC 4709, October 2006.
[RFC4918] Dusseault, L., "HTTP Extensions for Web Distributed
Authoring and Versioning (WebDAV)", RFC 4918, June 2007.
[RFC5661] Shepler, S., Eisler, M., and D. Noveck, "Network File [RFC5661] Shepler, S., Eisler, M., and D. Noveck, "Network File
System (NFS) Version 4 Minor Version 1 Protocol", System (NFS) Version 4 Minor Version 1 Protocol",
RFC 5661, January 2010. RFC 5661, January 2010.
[RFC5849] Hammer-Lahav, E., "The OAuth 1.0 Protocol", RFC 5849, [RFC5849] Hammer-Lahav, E., "The OAuth 1.0 Protocol", RFC 5849,
April 2010. April 2010.
[RFC6392] Alimi, R., Rahman, A., and Y. Yang, "A Survey of In- [RFC6392] Alimi, R., Rahman, A., and Y. Yang, "A Survey of In-
Network Storage Systems", RFC 6392, October 2011. Network Storage Systems", RFC 6392, October 2011.
[I-D.ietf-decade-problem-statement] [I-D.ietf-decade-problem-statement]
Song, H., Zong, N., Yang, Y., and R. Alimi, "DECoupled Song, H., Zong, N., Yang, Y., and R. Alimi, "DECoupled
Application Data Enroute (DECADE) Problem Statement", Application Data Enroute (DECADE) Problem Statement",
draft-ietf-decade-problem-statement-05 (work in progress), draft-ietf-decade-problem-statement-06 (work in progress),
February 2012. May 2012.
[I-D.ietf-decade-reqs] [I-D.ietf-decade-reqs]
Yingjie, G., Bryan, D., Yang, Y., and R. Alimi, "DECADE Yingjie, G., Bryan, D., Yang, Y., and R. Alimi, "DECADE
Requirements", draft-ietf-decade-reqs-05 (work in Requirements", draft-ietf-decade-reqs-06 (work in
progress), October 2011. progress), March 2012.
[GoogleStorageDevGuide]
"Google Storage Developer Guide", <http://code.google.com/
apis/storage/docs/developer-guide.html>.
[OpenFlow] [OpenFlow]
"OpenFlow Organization", <http://www.openflow.org/>. "OpenFlow Organization", <http://www.openflow.org/>.
[GoogleFileSystem] [GoogleFileSystem]
Ghemawat, S., Gobioff, H., and S. Leung, "The Google File Ghemawat, S., Gobioff, H., and S. Leung, "The Google File
System", SOSP 2003, October 2003. System", SOSP 2003, October 2003.
Appendix A. In-Network Storage Components Mapped to DECADE Architecture Appendix A. In-Network Storage Components Mapped to DECADE Architecture
In this section we evaluate how the basic components of an in-network In this section we evaluate how the basic components of an in-network
storage system identified in Section 3 of [RFC6392] map into a storage system identified in Section 3 of [RFC6392] map into a
DECADE-compatible system. DECADE-compatible system.
It is important to note that complex and/or application-specific
behavior is delegated to applications instead of tuning the storage
system wherever possible.
A.1. Data Access Interface A.1. Data Access Interface
Users can read and write objects of arbitrary size through the Clients can read and write objects of arbitrary size through the
Client's Data Controller, making use of a SDT. client's Data Controller, making use of a SDT.
A.2. Data Management Operations A.2. Data Management Operations
Users can move or delete previously stored objects via the Client's Clients can move or delete previously stored objects via the client's
Data Controller, making use of a SDT. Data Controller, making use of a SDT.
A.3. Data Search Capability A.3. Data Search Capability
Users can enumerate or search contents of Servers to find objects Clients can enumerate or search contents of servers to find objects
matching desired criteria through services provided by the Content matching desired criteria through services provided by the Content
Distribution Application (e.g., buffer-map exchanges, a DHT, or peer- Distribution Application (e.g., buffer-map exchanges, a DHT, or peer-
exchange). In doing so, End-Points may consult their local Data exchange). In doing so, Application End-Points might consult their
Index in the Client's Data Controller. local Data Index in the client's Data Controller.
A.4. Access Control Authorization A.4. Access Control Authorization
All methods of access control are supported: public-unrestricted, All methods of access control are supported: public-unrestricted,
public-restricted and private. Access Control Policies are generated public-restricted and private. Access Control Policies are generated
by a Content Distribution Application and provided to the Client's by a Content Distribution Application and provided to the client's
Resource Controller. The Server is responsible for implementing the Resource Controller. The server is responsible for implementing the
access control checks. access control checks.
A.5. Resource Control Interface A.5. Resource Control Interface
Users can manage the resources (e.g. bandwidth) on the DECADE server Clients can manage the resources (e.g. bandwidth) on the DECADE
that can be used by other Application End-Points. Resource Sharing server that can be used by other Application End-Points. Resource
Policies are generated by a Content Distribution Application and Sharing Policies are generated by a Content Distribution Application
provided to the Client's Resource Controller. The Server is and provided to the client's Resource Controller. The server is
responsible for implementing the resource sharing policies. responsible for implementing the resource sharing policies.
A.6. Discovery Mechanism A.6. Discovery Mechanism
The particular protocol used for discovery is outside the scope of The particular protocol used for discovery is outside the scope of
this document. However, options and considerations have been this document. However, options and considerations have been
discussed in Section 5.5. discussed in Section 5.5.
A.7. Storage Mode A.7. Storage Mode
Servers provide an object-based storage mode. Immutable data objects Servers provide an object-based storage mode. Immutable data objects
may be stored at a Server. Applications may consider existing blocks might be stored at a server. Applications might consider existing
as data objects, or they may adjust block sizes before storing in a blocks as data objects, or they might adjust block sizes before
Server. storing in a server.
Authors' Addresses Authors' Addresses
Richard Alimi Richard Alimi
Google Google
Email: ralimi@google.com Email: ralimi@google.com
Akbar Rahman Akbar Rahman
InterDigital Communications, LLC InterDigital Communications, LLC
 End of changes. 203 change blocks. 
785 lines changed or deleted 601 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/