draft-ietf-decade-arch-02.txt   draft-ietf-decade-arch-03.txt 
DECADE R. Alimi DECADE R. Alimi
Internet-Draft Google Internet-Draft Google
Intended status: Informational Y. Yang Intended status: Informational Y. Yang
Expires: January 12, 2012 Yale University Expires: March 31, 2012 Yale University
A. Rahman A. Rahman
InterDigital Communications, LLC InterDigital Communications, LLC
D. Kutscher D. Kutscher
NEC NEC
H. Liu H. Liu
Yale University Yale University
July 11, 2011 September 28, 2011
DECADE Architecture DECADE Architecture
draft-ietf-decade-arch-02 draft-ietf-decade-arch-03
Abstract Abstract
Content Distribution Applications (e.g., P2P applications) are widely Content Distribution Applications (e.g., P2P applications) are widely
used on the Internet and make up a large portion of the traffic in used on the Internet and make up a large portion of the traffic in
many networks. One technique to improve the network efficiency of many networks. One technique to improve the network efficiency of
these applications is to introduce storage capabilities within the these applications is to introduce storage capabilities within the
networks. This document presents an architecture, discusses the networks; this is the capability to be provided by DECADE (DECoupled
underlying principles, and identifies core components and protocols Application Data Enroute). This document presents an architecture
for supporting in-network storage functionality for these for DECADE, discusses the underlying principles, and identifies core
applications. components and protocols for supporting in-network storage
functionality for these applications.
Requirements Language Requirements Language
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [RFC2119]. document are to be interpreted as described in RFC 2119 [RFC2119].
Status of this Memo Status of this Memo
This Internet-Draft is submitted to IETF in full conformance with the This Internet-Draft is submitted to IETF in full conformance with the
skipping to change at page 2, line 10 skipping to change at page 2, line 10
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
The list of current Internet-Drafts can be accessed at The list of current Internet-Drafts can be accessed at
http://www.ietf.org/ietf/1id-abstracts.txt. http://www.ietf.org/ietf/1id-abstracts.txt.
The list of Internet-Draft Shadow Directories can be accessed at The list of Internet-Draft Shadow Directories can be accessed at
http://www.ietf.org/shadow.html. http://www.ietf.org/shadow.html.
This Internet-Draft will expire on January 12, 2012. This Internet-Draft will expire on March 31, 2012.
Copyright Notice Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the Copyright (c) 2011 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(http://trustee.ietf.org/license-info) in effect on the date of (http://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 3, line 15 skipping to change at page 3, line 15
Table of Contents Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 5
2. Functional Entities . . . . . . . . . . . . . . . . . . . . . 6 2. Functional Entities . . . . . . . . . . . . . . . . . . . . . 6
2.1. DECADE Server . . . . . . . . . . . . . . . . . . . . . . 6 2.1. DECADE Server . . . . . . . . . . . . . . . . . . . . . . 6
2.2. DECADE Client . . . . . . . . . . . . . . . . . . . . . . 6 2.2. DECADE Client . . . . . . . . . . . . . . . . . . . . . . 6
2.3. DECADE Storage Provider . . . . . . . . . . . . . . . . . 6 2.3. DECADE Storage Provider . . . . . . . . . . . . . . . . . 6
2.4. DECADE Content Provider . . . . . . . . . . . . . . . . . 6 2.4. DECADE Content Provider . . . . . . . . . . . . . . . . . 6
2.5. DECADE Content Consumer . . . . . . . . . . . . . . . . . 7 2.5. DECADE Content Consumer . . . . . . . . . . . . . . . . . 7
2.6. Content Distribution Application . . . . . . . . . . . . . 7 2.6. Content Distribution Application . . . . . . . . . . . . . 7
2.6.1. Application End-Point . . . . . . . . . . . . . . . . 7
3. Protocol Flow . . . . . . . . . . . . . . . . . . . . . . . . 7 3. Protocol Flow . . . . . . . . . . . . . . . . . . . . . . . . 7
3.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.2. An Example . . . . . . . . . . . . . . . . . . . . . . . . 9 3.2. An Example . . . . . . . . . . . . . . . . . . . . . . . . 9
4. Architectural Principles . . . . . . . . . . . . . . . . . . . 9 4. Architectural Principles . . . . . . . . . . . . . . . . . . . 9
4.1. Decoupled Control/Metadata and Data Planes . . . . . . . . 10 4.1. Decoupled Control/Metadata and Data Planes . . . . . . . . 10
4.2. Immutable Data Objects . . . . . . . . . . . . . . . . . . 11 4.2. Immutable Data Objects . . . . . . . . . . . . . . . . . . 11
4.3. Data Object Identifiers . . . . . . . . . . . . . . . . . 12 4.3. Data Object Identifiers . . . . . . . . . . . . . . . . . 12
4.4. Explicit Control . . . . . . . . . . . . . . . . . . . . . 12 4.4. Explicit Control . . . . . . . . . . . . . . . . . . . . . 12
4.5. Resource and Data Access Control through User 4.5. Resource and Data Access Control through User
Delegation . . . . . . . . . . . . . . . . . . . . . . . . 12 Delegation . . . . . . . . . . . . . . . . . . . . . . . . 12
4.5.1. Resource Allocation . . . . . . . . . . . . . . . . . 12
4.5.2. User Delegations . . . . . . . . . . . . . . . . . . . 13
5. System Components . . . . . . . . . . . . . . . . . . . . . . 13 5. System Components . . . . . . . . . . . . . . . . . . . . . . 13
5.1. Content Distribution Application . . . . . . . . . . . . . 14 5.1. Content Distribution Application . . . . . . . . . . . . . 14
5.1.1. Data Assembly . . . . . . . . . . . . . . . . . . . . 15
5.1.2. Native Protocols . . . . . . . . . . . . . . . . . . . 16
5.1.3. DECADE Client . . . . . . . . . . . . . . . . . . . . 16
5.2. DECADE Server . . . . . . . . . . . . . . . . . . . . . . 16 5.2. DECADE Server . . . . . . . . . . . . . . . . . . . . . . 16
5.2.1. Access Control . . . . . . . . . . . . . . . . . . . . 17
5.2.2. Resource Scheduling . . . . . . . . . . . . . . . . . 17
5.2.3. Data Store . . . . . . . . . . . . . . . . . . . . . . 18
5.3. Data Sequencing and Naming . . . . . . . . . . . . . . . . 18 5.3. Data Sequencing and Naming . . . . . . . . . . . . . . . . 18
5.3.1. DECADE Data Object Naming Scheme . . . . . . . . . . . 18
5.3.2. Application Usage . . . . . . . . . . . . . . . . . . 19
5.3.3. Application Usage Example . . . . . . . . . . . . . . 19
5.4. Token-based Authentication and Resource Control . . . . . 21 5.4. Token-based Authentication and Resource Control . . . . . 21
5.5. Discovery . . . . . . . . . . . . . . . . . . . . . . . . 22 5.5. Discovery . . . . . . . . . . . . . . . . . . . . . . . . 22
6. DECADE Protocols . . . . . . . . . . . . . . . . . . . . . . . 23 6. DECADE Protocols . . . . . . . . . . . . . . . . . . . . . . . 23
6.1. DECADE Resource Protocol (DRP) . . . . . . . . . . . . . . 23 6.1. DECADE Resource Protocol (DRP) . . . . . . . . . . . . . . 23
6.1.1. Controlled Resources . . . . . . . . . . . . . . . . . 23
6.1.2. Access and Resource Control Token . . . . . . . . . . 24
6.1.3. Status Information . . . . . . . . . . . . . . . . . . 25
6.1.4. Object Properties . . . . . . . . . . . . . . . . . . 26
6.2. Standard Data Transport (SDT) . . . . . . . . . . . . . . 26 6.2. Standard Data Transport (SDT) . . . . . . . . . . . . . . 26
6.2.1. Writing/Uploading Objects . . . . . . . . . . . . . . 26 7. Server-to-Server Protocols . . . . . . . . . . . . . . . . . . 29
6.2.2. Downloading Objects . . . . . . . . . . . . . . . . . 27
7. Server-to-Server Protocols . . . . . . . . . . . . . . . . . . 28
7.1. Operational Overview . . . . . . . . . . . . . . . . . . . 29 7.1. Operational Overview . . . . . . . . . . . . . . . . . . . 29
8. Potential Optimizations . . . . . . . . . . . . . . . . . . . 29 8. Potential Optimizations . . . . . . . . . . . . . . . . . . . 30
8.1. Pipelining to Avoid Store-and-Forward Delays . . . . . . . 30 8.1. Pipelining to Avoid Store-and-Forward Delays . . . . . . . 30
8.2. Deduplication . . . . . . . . . . . . . . . . . . . . . . 30 8.2. Deduplication . . . . . . . . . . . . . . . . . . . . . . 31
8.2.1. Traffic Deduplication . . . . . . . . . . . . . . . . 30 9. Security Considerations . . . . . . . . . . . . . . . . . . . 32
8.2.2. Cross-Server Storage Deduplication . . . . . . . . . . 31 10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 33
9. Security Considerations . . . . . . . . . . . . . . . . . . . 31 11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 33
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 32 11.1. Normative References . . . . . . . . . . . . . . . . . . . 33
11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 32 11.2. Informative References . . . . . . . . . . . . . . . . . . 33
11.1. Normative References . . . . . . . . . . . . . . . . . . . 32
11.2. Informative References . . . . . . . . . . . . . . . . . . 32
Appendix A. Appendix: Evaluation of Some Candidate Existing Appendix A. Appendix: Evaluation of Some Candidate Existing
Protocols for DECADE DRP and SDT . . . . . . . . . . 33 Protocols for DECADE DRP and SDT . . . . . . . . . . 34
A.1. HTTP . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 A.1. HTTP . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
A.1.1. HTTP Support for DECADE Resource Protocol A.2. WEBDAV . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Primitives . . . . . . . . . . . . . . . . . . . . . . 33 A.3. CDMI . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
A.1.2. HTTP Support for DECADE Standard Data Transport
Protocol Primitives . . . . . . . . . . . . . . . . . 34
A.1.3. Traffic De-duplication Primitives . . . . . . . . . . 35
A.1.4. Other Operations . . . . . . . . . . . . . . . . . . . 35
A.1.5. Conclusions . . . . . . . . . . . . . . . . . . . . . 35
A.2. WEBDAV . . . . . . . . . . . . . . . . . . . . . . . . . . 35
A.2.1. WEBDAV Support for DECADE Resource Protocol
Primitives . . . . . . . . . . . . . . . . . . . . . . 36
A.2.2. WebDAV Support for DECADE Standard Transport
Protocol Primitives . . . . . . . . . . . . . . . . . 37
A.2.3. Other Operations . . . . . . . . . . . . . . . . . . . 37
A.2.4. Conclusions . . . . . . . . . . . . . . . . . . . . . 38
Appendix B. In-Network Storage Components Mapped to DECADE Appendix B. In-Network Storage Components Mapped to DECADE
Architecture . . . . . . . . . . . . . . . . . . . . 39 Architecture . . . . . . . . . . . . . . . . . . . . 42
B.1. Data Access Interface . . . . . . . . . . . . . . . . . . 39 B.1. Data Access Interface . . . . . . . . . . . . . . . . . . 42
B.2. Data Management Operations . . . . . . . . . . . . . . . . 39 B.2. Data Management Operations . . . . . . . . . . . . . . . . 42
B.3. Data Search Capability . . . . . . . . . . . . . . . . . . 39 B.3. Data Search Capability . . . . . . . . . . . . . . . . . . 42
B.4. Access Control Authorization . . . . . . . . . . . . . . . 39 B.4. Access Control Authorization . . . . . . . . . . . . . . . 43
B.5. Resource Control Interface . . . . . . . . . . . . . . . . 39 B.5. Resource Control Interface . . . . . . . . . . . . . . . . 43
B.6. Discovery Mechanism . . . . . . . . . . . . . . . . . . . 39 B.6. Discovery Mechanism . . . . . . . . . . . . . . . . . . . 43
B.7. Storage Mode . . . . . . . . . . . . . . . . . . . . . . . 40 B.7. Storage Mode . . . . . . . . . . . . . . . . . . . . . . . 43
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 40 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . . 43
1. Introduction 1. Introduction
Content Distribution Applications are widely used on the Internet Content Distribution Applications are widely used on the Internet
today to distribute data, and they contribute a large portion of the today to distribute data, and they contribute a large portion of the
traffic in many networks. The DECADE architecture described in this traffic in many networks. The DECADE architecture described in this
document enables such applications to leverage in-network storage to document enables such applications to leverage in-network storage to
achieve more efficient content distribution. Specifically, in many achieve more efficient content distribution. Specifically, in many
subscriber networks, it can be expensive to upgrade network equipment subscriber networks, it can be expensive to upgrade network equipment
in the "last-mile", because it can involve replacing equipment and in the "last-mile", because it can involve replacing equipment and
skipping to change at page 5, line 32 skipping to change at page 5, line 32
This document presents an architecture for providing in-network This document presents an architecture for providing in-network
storage that can be integrated into Content Distribution storage that can be integrated into Content Distribution
Applications. The primary focus is P2P-based content distribution, Applications. The primary focus is P2P-based content distribution,
but the architecture may be useful to other applications with similar but the architecture may be useful to other applications with similar
characteristics and requirements. See [I-D.ietf-decade-reqs] for a characteristics and requirements. See [I-D.ietf-decade-reqs] for a
definition of the target applications supported by DECADE. definition of the target applications supported by DECADE.
The design philosophy of the DECADE architecture is to provide only The design philosophy of the DECADE architecture is to provide only
the core functionalities that are needed for applications to make use the core functionalities that are needed for applications to make use
of in-network storage. With such core functionalities, the protocol of in-network storage. With such core functionalities, DECADE may be
may be simpler and easier to support by storage providers. If more simpler and easier to support by storage providers. If more complex
complex functionalities are needed by a certain application or class functionalities are needed by a certain application or class of
of applications, it may be layered on top of the DECADE protocol. applications, it may be layered on top of DECADE.
The DECADE protocol will leverage existing data transport and DECADE will leverage existing data transport and application-layer
application layer protocols. The design is to work with a small set protocols. The design is to work with a small set of alternative
of alternative IETF protocols. In this document, we use "data IETF protocols. In this document, we use "data transport" to refer
transport" to refer to a protocol that is used to read data from and to a protocol that is used to read data from and write data into
write data into DECADE in-network storage. DECADE in-network storage.
This document proceeds in two steps. First, it details the core This document proceeds in two steps. First, it details the core
architectural principles that we use to guide the DECADE design. architectural principles that we use to guide the DECADE design.
Next, given these core principles, this document presents the core Next, given these core principles, this document presents the core
components of the DECADE architecture and identifies the usage of components of the DECADE architecture and identifies the usage of
existing protocols and where there is a need for new protocol existing protocols and where there is a need for new protocol
development. development.
This document does not define any new protocol. Instead, when
identifying the need for a new protocol, it presents an abstract
design including the necessary functions of that protocol (including
rationale) in order to guide future protocol development.
2. Functional Entities 2. Functional Entities
This section defines the functional entities involved in a DECADE This section defines the functional entities involved in a DECADE
system. Functional entities can be classified as follows: system. Functional entities can be classified as follows:
o A physical or logical component in the DECADE architecture: DECADE o A physical or logical component in the DECADE architecture: DECADE
Client, DECADE Server, Content Distribution Application and Client, DECADE Server, Content Distribution Application and
Application End Point; Application End Point;
o Operator of a physical or logical component in the DECADE o Operator of a physical or logical component in the DECADE
skipping to change at page 8, line 34 skipping to change at page 8, line 34
| | | | | | | |
| | | | | | | |
v V v V v V v V
.=============. DRP .=============. .=============. DRP .=============.
| DECADE | <------------------> | DECADE | | DECADE | <------------------> | DECADE |
| Server | <------------------> | Server | | Server | <------------------> | Server |
`=============' SDT `=============' `=============' SDT `============='
Figure 1: Generic Protocol Flow Figure 1: Generic Protocol Flow
Second, Standard Data Transport protocols (e.g., WebDAV or NFS or Second, Standard Data Transport protocols (e.g., HTTP, WebDAV, or
HTTP/s) are used to transfer data objects to and from a DECADE CDMI) are used to transfer data objects to and from a DECADE Server.
Server. The DECADE architecture may be used with multiple standard The DECADE architecture may be used with multiple standard data
data transports. transports.
Decoupling the protocols in this way allows DECADE to directly Decoupling the protocols in this way allows DECADE to directly
utilize existing standard data transports, as well as allowing both utilize existing standard data transports, as well as allowing both
DECADE and DRP to evolve independently from data transports. DECADE and DRP to evolve independently from data transports.
It is also important to note that the two protocols do not need to be It is also important to note that the two protocols do not need to be
separate on the wire. For example, DRP messages may be piggybacked separate on the wire. For example, DRP messages may be piggybacked
within some extension fields provided by certain data transport within some extension fields provided by certain data transport
protocols. In such a scenario, DRP is technically a data structure protocols. In such a scenario, DRP is technically a data structure
(transported by other protocols), but it can still be considered as a (transported by other protocols), but it can still be considered as a
skipping to change at page 9, line 28 skipping to change at page 9, line 28
When an Application End-Point wishes to use its DECADE storage When an Application End-Point wishes to use its DECADE storage
server, it provides a token (see Section 6.1.2 for details) to the server, it provides a token (see Section 6.1.2 for details) to the
other Application End-Point. The token is sent using the Content other Application End-Point. The token is sent using the Content
Distribution Application's native protocol. Distribution Application's native protocol.
The steps of the example are illustrated in Figure 2. First, B The steps of the example are illustrated in Figure 2. First, B
requests a data object from A using their native protocol. Next, A requests a data object from A using their native protocol. Next, A
uses the DECADE Resource Protocol (DRP) to obtain a token from its uses the DECADE Resource Protocol (DRP) to obtain a token from its
DECADE storage server, S(A). A then provides the token to B (again, DECADE storage server, S(A). A then provides the token to B (again,
using their native protocol). Finally, provides the token to S(B) using their native protocol). Finally, B provides the token to S(A)
via DRP, and requests and downloads the data object via a Standard via DRP, and requests and downloads the data object via a Standard
Data Transport (SDT). Data Transport (SDT).
.----------. .----------.
----------> | S(A) | <------ ----------> | S(A) | <------
2. Obtain / `----------' \ 4. Request and 2. Obtain / `----------' \ 4. Request and
Token / \ Download Object Token / \ Download Object
(DRP) / \ (DRP + SDT) (DRP) / \ (DRP + SDT)
v 1. App request v v 1. App request v
.-------------. <--------------------------- .-------------. .-------------. <--------------------------- .-------------.
skipping to change at page 10, line 42 skipping to change at page 10, line 42
Given the diversity of control-plane functions, in-network storage Given the diversity of control-plane functions, in-network storage
should export basic mechanisms and allow as much flexibility as should export basic mechanisms and allow as much flexibility as
possible to the control planes to implement specific policies. This possible to the control planes to implement specific policies. This
conforms to the end-to-end systems principle and allows innovation conforms to the end-to-end systems principle and allows innovation
and satisfaction of specific business goals. and satisfaction of specific business goals.
Decoupling control plane and data plane is not new. For example, Decoupling control plane and data plane is not new. For example,
OpenFlow is an implementation of this principle for Internet routing, OpenFlow is an implementation of this principle for Internet routing,
where the computation of the forwarding table and the application of where the computation of the forwarding table and the application of
the forwarding table are separated. Google File System applies the the forwarding table are separated. Google File System
principle to file system design, by utilizing the Master to handle [GoogleFileSystem] applies the principle to file system design, by
the meta-data management, and the Chunk Servers to handle the data utilizing the Master to handle the meta-data management, and the
plane functions (i.e., read and write of chunks of data). NFS4 also Chunk Servers to handle the data plane functions (i.e., read and
write of chunks of data). NFSv4.1's pNFS extension [RFC5661] also
implements this principle. implements this principle.
Note that applications may have different Data Plane implementations Note that applications may have different Data Plane implementations
in order to support particular requirements (e.g., low latency). In in order to support particular requirements (e.g., low latency). In
order to provide interoperability, the DECADE architecture does not order to provide interoperability, the DECADE architecture does not
intend to enable arbitrary data transport protocols. However, the intend to enable arbitrary data transport protocols. However, the
architecture may allow for more-than-one data transport protocols to architecture may allow for more-than-one data transport protocols to
be used. be used.
Also note that although an application's existing control plane Also note that although an application's existing control plane
skipping to change at page 13, line 24 skipping to change at page 13, line 26
considerations in certain deployment scenarios. considerations in certain deployment scenarios.
To provide a scalable way to manage applications granted resources at To provide a scalable way to manage applications granted resources at
a DECADE server, we consider an architecture that adds a layer of a DECADE server, we consider an architecture that adds a layer of
indirection. Instead of granting resources to an application, the indirection. Instead of granting resources to an application, the
DECADE server grants a share of the resources to a user. The user DECADE server grants a share of the resources to a user. The user
may in turn share the granted resources amongst multiple may in turn share the granted resources amongst multiple
applications. The share of resources granted by a storage provider applications. The share of resources granted by a storage provider
is called a User Delegation. is called a User Delegation.
As a simple example, DECADE Server operated by an ISP may be As a simple example, a DECADE Server operated by an ISP may be
configured to grant each ISP Subscriber 1.5 Mbps of bandwidth. The configured to grant each ISP Subscriber 1.5 Mbps of bandwidth. The
ISP Subscriber may in turn divide this share of resources amongst a ISP Subscriber may in turn divide this share of resources amongst a
video streaming application and file-sharing application which are video streaming application and file-sharing application which are
running concurrently. running concurrently.
In general, a User Delegation may be granted to an end-user (e.g., an In general, a User Delegation may be granted to an end-user (e.g., an
ISP subscriber), a Content Provider, or an Application Provider. A ISP subscriber), a Content Provider, or an Application Provider. A
particular instance of an application may make use of the storage particular instance of an application may make use of the storage
resources: resources:
skipping to change at page 14, line 18 skipping to change at page 14, line 22
Content Distribution Applications have many functional components. Content Distribution Applications have many functional components.
For example, many P2P applications have components and algorithms to For example, many P2P applications have components and algorithms to
manage overlay topology management, piece selection, etc. In manage overlay topology management, piece selection, etc. In
supporting DECADE, it may be advantageous for an application supporting DECADE, it may be advantageous for an application
developer to consider DECADE in the implementation of these developer to consider DECADE in the implementation of these
components. However, in this architecture document, we focus on the components. However, in this architecture document, we focus on the
components directly employed to support DECADE. components directly employed to support DECADE.
Figure 3 illustrates the components discussed in this section from Figure 3 illustrates the components discussed in this section from
the perspective of a single Application End-Point and their relation the perspective of a single Application End-Point and their relation
to the DECADE protocols. to DECADE.
Native Protocol(s) Native Protocol(s)
(with other Application End-Points) (with other Application End-Points)
.---------------------> .--------------------->
| |
| |
.----------------------------------------------------------. .----------------------------------------------------------.
| Application End-Point | | Application End-Point |
| .------------. .-------------------. | | .------------. .-------------------. |
| | App-Layer | ... | App Data Assembly | | | | App-Layer | ... | App Data Assembly | |
skipping to change at page 18, line 37 skipping to change at page 18, line 37
properties: properties:
o Simple integrity verification o Simple integrity verification
o Unique names (with high probability) o Unique names (with high probability)
o Application independent, without a new IANA-maintained registry o Application independent, without a new IANA-maintained registry
The DECADE naming scheme also includes a "type" field, the "type" The DECADE naming scheme also includes a "type" field, the "type"
identifier indicates that the name is the hash of the data object's identifier indicates that the name is the hash of the data object's
content and the particular hashing algorithm used. This allows the content and the particular hashing algorithm used. This allows
DECADE protocol to evolve by either changing the hashing algorithm DECADE to evolve by either changing the hashing algorithm (e.g., if
(e.g., if security vulnerabilities with an existing hashing algorithm security vulnerabilities with an existing hashing algorithm are
are discovered), or move to a different naming scheme altogether. discovered), or moving to a different naming scheme altogether.
The specific format of the name (e.g., encoding, hash algorithms, The specific format of the name (e.g., encoding, hash algorithms,
etc) is out of scope of this document, and left for protocol etc) is out of scope of this document, and left for protocol
specification. specification.
Another advantage of this scheme is that a DECADE client knows the Another advantage of this scheme is that a DECADE client knows the
name of a data object before it is completely stored at the DECADE name of a data object before it is completely stored at the DECADE
server. This allows for particular optimizations, such as server. This allows for particular optimizations, such as
advertising data object while the data object is being stored, advertising data object while the data object is being stored,
removing store-and-forward delays. For example, a DECADE client A removing store-and-forward delays. For example, a DECADE client A
skipping to change at page 22, line 25 skipping to change at page 22, line 25
o Tokens can provide anonymous access, in which a DECADE Server does o Tokens can provide anonymous access, in which a DECADE Server does
not need to know the identity of each DECADE Client that accesses not need to know the identity of each DECADE Client that accesses
it. This enables a DECADE Client to send tokens to DECADE Clients it. This enables a DECADE Client to send tokens to DECADE Clients
in other administrative or security domains, and allow them to in other administrative or security domains, and allow them to
read or write data objects from its DECADE storage. read or write data objects from its DECADE storage.
It is important to note that, in addition to DECADE Clients applying It is important to note that, in addition to DECADE Clients applying
access control policies to DECADE data objects, the DECADE Server may access control policies to DECADE data objects, the DECADE Server may
be configured to apply additional policies based on user, object, be configured to apply additional policies based on user, object,
geographic location, etc. Defining such policies is out of scope of geographic location, etc. Defining such policies is out of scope for
the DECADE Working Group, but in such a case, a DECADE Client may be DECADE, but in such a case, a DECADE Client may be denied access even
denied access even though it possess a valid token. though it possess a valid token.
5.5. Discovery 5.5. Discovery
DECADE includes a discovery mechanism through which DECADE clients DECADE includes a discovery mechanism through which DECADE clients
locate an appropriate DECADE Server. [I-D.ietf-decade-reqs] details locate an appropriate DECADE Server. [I-D.ietf-decade-reqs] details
specific requirements of the discovery mechanism; this section specific requirements of the discovery mechanism; this section
discusses how they relate to other principles outlined in this discusses how they relate to other principles outlined in this
document. document.
A discovery mechanism allows a DECADE client to determine an IP A discovery mechanism allows a DECADE client to determine an IP
skipping to change at page 23, line 13 skipping to change at page 23, line 13
It is important to note that the discovery mechanism outlined here It is important to note that the discovery mechanism outlined here
does not provide the ability to locate arbitrary DECADE servers to does not provide the ability to locate arbitrary DECADE servers to
which a DECADE client might obtain tokens from others. To do so which a DECADE client might obtain tokens from others. To do so
requires application-level knowledge, and it is assumed that this requires application-level knowledge, and it is assumed that this
functionality is implemented in the Content Distribution Application, functionality is implemented in the Content Distribution Application,
or if desired and needed, as an extension to this DECADE or if desired and needed, as an extension to this DECADE
architecture. architecture.
6. DECADE Protocols 6. DECADE Protocols
This section specifies the DECADE Resource Protocol (DRP) and the This section presents the DECADE Resource Protocol (DRP) and the
Standard Data Transport (SDT) in terms of abstract protocol Standard Data Transport (SDT) in terms of abstract protocol
interactions that are intended to mapped to specific protocols. Note interactions that are intended to mapped to specific protocols. Note
that while the protocols are logically separate, DRP is specified as that while the protocols are logically separate, DRP is specified as
being carried through extension fields within an SDT (e.g., HTTP being carried through extension fields within an SDT (e.g., HTTP
headers). headers).
The DRP is the protocol used by a DECADE client to configure the The DRP is the protocol used by a DECADE client to configure the
resources and authorization used to satisfy requests (reading, resources and authorization used to satisfy requests (reading,
writing, and management operations concerning DECADE objects) at a writing, and management operations concerning DECADE objects) at a
DECADE server. The SDT is used to send the operations to the DECADE DECADE server. The SDT is used to send the operations to the DECADE
skipping to change at page 26, line 5 skipping to change at page 26, line 5
elements can be requested: elements can be requested:
* what objects have been accessed how many times * what objects have been accessed how many times
* access tokens that a server as seen for a given object * access tokens that a server as seen for a given object
The DECADE server can decide on time bounds for which this The DECADE server can decide on time bounds for which this
information is stored and specify the corresponding time frame in information is stored and specify the corresponding time frame in
the response to such requests. the response to such requests.
6.1.4. Object Properties 6.1.4. Object Attributes
Objects that are stored on a DECADE server can provide properties (in Objects that are stored on a DECADE server may have associated
addition to the object identifier and the actual content). Depending attributes (in addition to the object identifier and the actual
on authorization, DECADE clients may get or set such properties. content) that relate to the data storage and its management. These
This authorization (and the mapping to application contexts) is based attributes may be used by the DECADE server (and possibly the
on the user delegation concept as described in Section 4.5. The underlying storage system) to perform specialized processing or
DECADE architecture does not limit the set of permissible properties, handling for the data object, or to attach related DECADE server or
but rather specifies a set of baseline properties that SHOULD be storage-layer properties to the data object. These attributes have a
supported by implementations. scope local to a DECADE server. In particular, these attributes are
not applied to a DECADE server or client to which a data object is
copied.
Depending on authorization, DECADE clients may get or set such
attributes. This authorization (and the mapping to application
contexts) is based on the user delegation concept as described in
Section 4.5. The DECADE architecture does not limit the set of
permissible attributes, but rather specifies a set of baseline
attributes that SHOULD be supported by implementations.
Suggested attributes are the following:
TTL: TTL of the object as an absolute time value TTL: TTL of the object as an absolute time value
object size: in bytes object size: in bytes
MIME type MIME type
access statistics: how often the object has been accessed (and what access statistics: how often the object has been accessed (and what
tokens have been used) tokens have been used)
It is important to note that the Object Attributes defined here are
distinct from application metadata (see Section 4.1). Application
metadata is custom information that an application may wish to
associate with a data object to understand its semantic meaning
(e.g., whether it is video and/or audio, its playback length in time,
or its index in a stream). If an application wishes to store such
metadata persistently within DECADE, it can be stored within data
objects themselves.
6.2. Standard Data Transport (SDT) 6.2. Standard Data Transport (SDT)
A DECADE server provide a data access interface, and SDT is used to A DECADE server provide a data access interface, and SDT is used to
write objects to a server and to read (download) objects from a write objects to a server and to read (download) objects from a
server. Semantically, SDT is a client-server protocol, i.e., the server. Semantically, SDT is a client-server protocol, i.e., the
DECADE server always responds to client requests. DECADE server always responds to client requests.
An SDT used in DECADE SHOULD offer a transport mode that provides An SDT used in DECADE SHOULD offer a transport mode that provides
confidentiality and integrity. confidentiality and integrity.
skipping to change at page 27, line 34 skipping to change at page 28, line 8
Description: The PUT method is used by a DECADE client to upload an Description: The PUT method is used by a DECADE client to upload an
object with an associated name 'NAME' to a DECADE server. object with an associated name 'NAME' to a DECADE server.
RESPONSES: The DECADE server MUST respond with one the following RESPONSES: The DECADE server MUST respond with one the following
response messages: response messages:
CREATED: The object has been uploaded successfully and is now CREATED: The object has been uploaded successfully and is now
available under the specified name. available under the specified name.
ERRORs: There was an error uploading the content ERRORs:
VALIDATION_FAILED: The contents of the data object received by
the DECADE server did not match the provided name (i.e.,
hash validation failed).
PERMISSION_DENIED: The DECADE client lacked sufficient
permission to store the object.
Specifics regarding error handling, including additional error
conditions, precedence for returned errors and its relation
with server policy, are deferred to eventual protocol
specification.
6.2.2. Downloading Objects 6.2.2. Downloading Objects
A DECADE client can request named objects from a DECADE server. In A DECADE client can request named objects from a DECADE server. In
the following, we provide an abstract specification of the download the following, we provide an abstract specification of the download
operation that we name 'GET METHOD'. See Section 5.3 for an example operation that we name 'GET METHOD'. See Appendix A.1 for an example
how this could be mapped to HTTP. how this could be mapped to HTTP.
Method GET: Method GET:
Parameters: Parameters:
NAME: The naming of the object according to Section 5.3. NAME: The naming of the object according to Section 5.3.
Description: The GET method is used by a DECADE client to download Description: The GET method is used by a DECADE client to download
an object with an associated name 'NAME' from a DECADE server. an object with an associated name 'NAME' from a DECADE server.
RESPONSES: The DECADE server MUST respond with one the following RESPONSES: The DECADE server MUST respond with one the following
response messages: response messages:
OK: The request has succeeded, and an entity corresponding to the OK: The request has succeeded, and an entity corresponding to the
requested resource is sent in the response. requested resource is sent in the response.
ERRORs: ERRORs:
NOTFOUND: The DECADE server has not found anything matching NOT_FOUND: The DECADE server has not found anything matching
the request object name. the request object name.
Other Errors: TBD in a future version of this document PERMISSION_DENIED: The DECADE client lacked sufficient
permission to read the object.
NOT_AVAILABLE: The data object exists but is currently
unavailable for download (e.g., due to server policy).
Specifics regarding error handling, including additional error
conditions, precedence for returned errors and its relation
with server policy, are defered to eventual protocol
specification.
7. Server-to-Server Protocols 7. Server-to-Server Protocols
An important feature of DECADE is the capability for one DECADE An important feature of DECADE is the capability for one DECADE
server to directly download data objects from another DECADE server. server to directly download data objects from another DECADE server.
This capability allows Applications to directly replicate data This capability allows Applications to directly replicate data
objects between servers without requiring end-hosts to use uplink objects between servers without requiring end-hosts to use uplink
capacity to upload data objects to a different DECADE server. capacity to upload data objects to a different DECADE server.
Similar to other operations in DRP and SDT, replicating data objects Similar to other operations in DRP and SDT, replicating data objects
between DECADE servers is an explicit operation. between DECADE servers is an explicit operation.
skipping to change at page 31, line 34 skipping to change at page 32, line 34
How to integrate traffic deduplication with HTTP is shown in How to integrate traffic deduplication with HTTP is shown in
Appendix A.1.3. Appendix A.1.3.
8.2.2. Cross-Server Storage Deduplication 8.2.2. Cross-Server Storage Deduplication
The same object might be uploaded multiple times to different DECADE The same object might be uploaded multiple times to different DECADE
servers. For storage efficiency, storage providers may desire that a servers. For storage efficiency, storage providers may desire that a
single object be stored on one or a few servers. They might single object be stored on one or a few servers. They might
implement an internal mechanism to achieve the goal, for example, by implement an internal mechanism to achieve the goal, for example, by
redirecting requests to proper servers. The DECADE protocol supports redirecting requests to proper servers. DECADE supports the
the redirection of DECADE client requests to support further cross- redirection of DECADE client requests to support further cross-server
server storage deduplication. storage deduplication.
9. Security Considerations 9. Security Considerations
In general, the security considerations mentioned in In general, the security considerations mentioned in
[I-D.ietf-decade-problem-statement] apply to this document as well. [I-D.ietf-decade-problem-statement] apply to this document as well.
In addition, it should be noted that the token-based approach In addition, it should be noted that the token-based approach
Section 5.4 provides authorization through token delegation. The Section 5.4 provides authorization through token delegation. The
strength of this authorization depends on several factors: strength of this authorization depends on several factors:
skipping to change at page 32, line 44 skipping to change at page 33, line 44
[RFC4331] Korver, B. and L. Dusseault, "Quota and Size Properties fo [RFC4331] Korver, B. and L. Dusseault, "Quota and Size Properties fo
r Distributed Authoring and Versioning (DAV) Collections", r Distributed Authoring and Versioning (DAV) Collections",
RFC 4331, February 2006. RFC 4331, February 2006.
[RFC4709] Reschke, J., "Mounting Web Distributed Authoring and [RFC4709] Reschke, J., "Mounting Web Distributed Authoring and
Versioning (WebDAV) Servers", RFC 4709, October 2006. Versioning (WebDAV) Servers", RFC 4709, October 2006.
[RFC4918] Dusseault, L., "HTTP Extensions for Web Distributed [RFC4918] Dusseault, L., "HTTP Extensions for Web Distributed
Authoring and Versioning (WebDAV)", RFC 4918, June 2007. Authoring and Versioning (WebDAV)", RFC 4918, June 2007.
[RFC5661] Shepler, S., Eisler, M., and D. Noveck, "Network File
System (NFS) Version 4 Minor Version 1 Protocol",
RFC 5661, January 2010.
[I-D.ietf-decade-problem-statement] [I-D.ietf-decade-problem-statement]
Song, H., Zong, N., Yang, Y., and R. Alimi, "DECoupled Song, H., Zong, N., Yang, Y., and R. Alimi, "DECoupled
Application Data Enroute (DECADE) Problem Statement", Application Data Enroute (DECADE) Problem Statement",
draft-ietf-decade-problem-statement-03 (work in progress), draft-ietf-decade-problem-statement-03 (work in progress),
March 2011. March 2011.
[I-D.ietf-decade-survey] [I-D.ietf-decade-survey]
Alimi, R., Rahman, A., and Y. Yang, "A Survey of In- Alimi, R., Rahman, A., and Y. Yang, "A Survey of In-
network Storage Systems", draft-ietf-decade-survey-04 network Storage Systems", draft-ietf-decade-survey-06
(work in progress), March 2011. (work in progress), August 2011.
[I-D.ietf-decade-reqs] [I-D.ietf-decade-reqs]
Yingjie, G., Bryan, D., Yang, Y., and R. Alimi, "DECADE Yingjie, G., Bryan, D., Yang, Y., and R. Alimi, "DECADE
Requirements", draft-ietf-decade-reqs-02 (work in Requirements", draft-ietf-decade-reqs-03 (work in
progress), May 2011. progress), July 2011.
[GoogleStorageDevGuide] [GoogleStorageDevGuide]
"Google Storage Developer Guide", <http://code.google.com/ "Google Storage Developer Guide", <http://code.google.com/
apis/storage/docs/developer-guide.html>. apis/storage/docs/developer-guide.html>.
[GoogleFileSystem]
Ghemawat, S., Gobioff, H., and S. Leung, "The Google File
System", SOSP 2003, October 2003.
[CDMI] "CDMI", <http://www.snia.org/cdmi>.
Appendix A. Appendix: Evaluation of Some Candidate Existing Protocols Appendix A. Appendix: Evaluation of Some Candidate Existing Protocols
for DECADE DRP and SDT for DECADE DRP and SDT
In this section we evaluate how well the abstract protocol In this section we evaluate how well the abstract protocol
interactions specified in Section 6 for DECADE DRP and SDT can be interactions specified in Section 6 for DECADE DRP and SDT can be
fulfilled by existing protocols such as HTTP and WEBDAV. fulfilled by existing protocols such as HTTP, WEBDAV, and CDMI.
A.1. HTTP A.1. HTTP
HTTP [RFC2616] is a key protocol for the Internet in general and HTTP [RFC2616] is a key protocol for the Internet in general and
especially for the World Wide Web. HTTP is a request-response especially for the World Wide Web. HTTP is a request-response
protocol. A typical transaction involves a client (e.g. web browser) protocol. A typical transaction involves a client (e.g. web browser)
requesting content (resources) from a web server. Another example is requesting content (resources) from a web server. Another example is
when a client stores or deletes content from a server. when a client stores or deletes content from a server.
A.1.1. HTTP Support for DECADE Resource Protocol Primitives A.1.1. HTTP Support for DECADE Resource Protocol Primitives
skipping to change at page 34, line 7 skipping to change at page 35, line 13
for the server, and then checking the authorization of a user before for the server, and then checking the authorization of a user before
it stores or retrieves content. HTTP supports a rudimentary access it stores or retrieves content. HTTP supports a rudimentary access
control via "HTTP Secure" (HTTPS). HTTPS is a combination of HTTP control via "HTTP Secure" (HTTPS). HTTPS is a combination of HTTP
with SSL/TLS. The main use of HTTPS is to authenticate the server with SSL/TLS. The main use of HTTPS is to authenticate the server
and encrypt all traffic between the client and the server. There is and encrypt all traffic between the client and the server. There is
also a mode to support client authentication though this is less also a mode to support client authentication though this is less
frequently used. frequently used.
A.1.1.2. Communication Resource Controls Primitives A.1.1.2. Communication Resource Controls Primitives
Communications resources include bandwidth (upload/download) and Communication resources include bandwidth (upload/download) and
number of simultaneous connected clients (connections). HTTP number of simultaneous connected clients (connections). HTTP
supports bandwidth control indirectly through "persistent" HTTP supports bandwidth control indirectly through "persistent" HTTP
connections. Persistent HTTP connections allows a client to keep connections. Persistent HTTP connections allows a client to keep
open the underlying TCP connection to the server to allow streaming open the underlying TCP connection to the server to allow streaming
and pipelining (multiple simultaneous requests for a given client). and pipelining (multiple simultaneous requests for a given client).
HTTP does not define protocol operation to allow limiting the HTTP does not define protocol operation to allow limiting the
communication resources to a client. However servers typically communication resources to a client. However servers typically
perform this function via implementation algorithms. perform this function via implementation algorithms.
skipping to change at page 35, line 13 skipping to change at page 36, line 20
supports downloading through the GET and HEAD methods. GET fetches a supports downloading through the GET and HEAD methods. GET fetches a
specific resource as identified by the URL. HEAD is similar but only specific resource as identified by the URL. HEAD is similar but only
fetches the metadata ("header") associated with the resource but not fetches the metadata ("header") associated with the resource but not
the resource itself. the resource itself.
A.1.3. Traffic De-duplication Primitives A.1.3. Traffic De-duplication Primitives
To challenge a remote entity for an object, the DECADE server should To challenge a remote entity for an object, the DECADE server should
provide a seed number, which is generated by the server randomly, and provide a seed number, which is generated by the server randomly, and
ask the remote entity to return a hash calculated from the seed ask the remote entity to return a hash calculated from the seed
number and the content of the object. The server MAY also specify number and the content of the object. The server may also specify
the hash function which the remote entity should use. HTTP supports the hash function which the remote entity should use. HTTP supports
the challenge message through the GET methods. The message type the challenge message through the GET methods. The message type
("challenge"), the seed number and the hash function name are put in ("challenge"), the seed number and the hash function name are put in
URL. In the reply, the hash is sent in an ETAG header. URL. In the reply, the hash is sent in an ETAG header.
A.1.4. Other Operations A.1.4. Other Operations
HTTP supports deleting of content on the server through the DELETE HTTP supports deleting of content on the server through the DELETE
method. method.
skipping to change at page 39, line 5 skipping to change at page 40, line 7
DECADE: DECADE:
- LOCK/UNLOCK - LOCK/UNLOCK
Finally, some extensions to WebDAV may still be required to meet all Finally, some extensions to WebDAV may still be required to meet all
DECADE requirements. For example, defining a new WebDAV "time-to- DECADE requirements. For example, defining a new WebDAV "time-to-
live" property may be useful for DECADE. Further analysis is live" property may be useful for DECADE. Further analysis is
required to fully define the potential extensions to WebDAV to meet required to fully define the potential extensions to WebDAV to meet
all DECADE requirements. all DECADE requirements.
A.3. CDMI
The Cloud Data Management Interface (CDMI) specification defines a
functional interface through which applications can store and manage
data objects in a cloud storage environment. The CDMI interface for
reading/writing data is based on standard HTTP requests, with CDMI-
specific encodings using JavaScript Object Notation (JSON). CDMI is
specified by the Storage Networking Industry Association (SNIA)
[CDMI].
A.3.1. CDMI Support for DECADE Resource Protocol Primitives
DRP provides configuration of access control and resource sharing
policies on DECADE servers.
A.3.1.1. Access Control Primitives
Access control includes mechanisms for defining the access policies
for the server, and then checking the authorization of a user before
it stores or retrieves content. CDMI defines an Access Control List
(ACL) per data object, and thus supports access control (read and/or
write) at the data object granularity. An ACL contains a set of
Access Control Entries (ACEs), where each ACE specifies a principal
(i.e. user or group of users) and a set of privileges that are
granted to that principal.
CDMI requires that an HTTP authentication mechanism be available for
the server to validate the identity of a principal (client).
Specifically, CDMI requires that either HTTP Basic Authentication or
HTTP Digest Authentication be supported. CDMI recommends that HTTP
over TLS (HTTPS) is supported to encrypt the data sent over the
network.
A.3.1.2. Communication Resource Controls Primitives
Communication resources include bandwidth (upload/download) and
number of simultaneous connected clients (connections). CDMI
supports two key data attributes which provide control over the
communication resources to a client: "cdmi_max_throughput" and
"cdmi_max_latency". These attributes are defined in the metadata for
data objects and indicate the desired bandwidth or delay for
transmission of the data object from the cloud server to the client.
A.3.1.3. Storage Resource Control Primitives
Storage resources include amount of quantity and lifetime of storage.
CDMI defines metadata for individual data objects and general storage
system configuration which can be used for storage resource control.
In particular, CDMI defines the following metadata fields:
- cdmi_data_redundancy: desired number of copies to be
maintained,
- cdmi_geographic_placement: region where object is permitted to
be stored,
- cdmi_retention_period: time interval object is to be retained,
and
- cdmi_retention_autodelete: whether object should be auto
deleted after retention period.
A.3.2. CDMI Support for DECADE Standard Transport Protocol Primitives
SDT is used to write objects and read (download) objects from a
DECADE server. The object can be either a self-contained object such
as a multimedia file or a chunk from a P2P system.
A.3.2.1. Writing Primitives
Writing involves uploading objects to the server. CDMI supports
standard HTTP methods for PUT and POST as described in
Appendix A.1.2.1.
A.3.2.2. Downloading Primitives
Downloading involves fetching of an object from the server. CDMI
supports the standard HTTP GET method as described in
Appendix A.1.2.2.
A.3.3. Other Operations
CDMI supports DELETE as described in Appendix A.1.4. CDMI also
supports COPY and MOVE operations.
CDMI supports the concept of containers of data objects to support
joint operations on related objects. For example, GET may be done on
a single data object or on an entire container.
CDMI supports a global naming scheme. Every object stored within a
CDMI system will have a globally unique object string identifier
(ObjectID) assigned at creation time.
A.3.4. Conclusions
CDMI has a rich array of features that can provide a good base for
DRP and SDT for DECADE. An initial analysis finds that the following
CDMI features may be useful for DECADE:
- access control
- storage resource control
- communication resource control
- COPY/MOVE operations
- data containers
- naming scheme
Appendix B. In-Network Storage Components Mapped to DECADE Architecture Appendix B. In-Network Storage Components Mapped to DECADE Architecture
In this section we evaluate how the basic components of an in-network In this section we evaluate how the basic components of an in-network
storage system identified in Section 3 of [I-D.ietf-decade-survey] storage system identified in Section 3 of [I-D.ietf-decade-survey]
map into the DECADE architecture. map into the DECADE architecture.
It is important to note that complex and/or application-specific It is important to note that complex and/or application-specific
behavior is delegated to applications instead of tuning the storage behavior is delegated to applications instead of tuning the storage
system wherever possible. system wherever possible.
 End of changes. 43 change blocks. 
112 lines changed or deleted 249 lines changed or added

This html diff was produced by rfcdiff 1.41. The latest version is available from http://tools.ietf.org/tools/rfcdiff/