draft-ietf-ippm-npmps-00.txt   draft-ietf-ippm-npmps-01.txt 
Network Working Group V. Raisanen Network Working Group V. Raisanen
INTERNET-DRAFT Nokia INTERNET-DRAFT Nokia
Expiration Date: September 2000 G. Grotefeld Expiration Date: November 2000 G. Grotefeld
Motorola Motorola
March 2000 May 2000
Network performance measurement for periodic streams Network performance measurement for periodic streams
<draft-ietf-ippm-npmps-00.txt> <draft-ietf-ippm-npmps-01.txt>
1. Status of this Memo 1. Status of this Memo
This document is an Internet-Draft and is in full conformance with This document is an Internet-Draft and is in full conformance with
all provisions of Section 10 of RFC2026. all provisions of Section 10 of RFC2026.
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF), its areas, and its working groups. Note that Task Force (IETF), its areas, and its working groups. Note that
other groups may also distribute working documents as Internet- other groups may also distribute working documents as Internet-
Drafts. Drafts.
skipping to change at page 2, line 5 skipping to change at page 2, line 5
for use as bearer for multimedia streams. In this document, for use as bearer for multimedia streams. In this document,
the reader is assumed to be familiar with the terminology of the the reader is assumed to be familiar with the terminology of the
Framework for IP Performance Metrics RFC 2330 [1]. This document is Framework for IP Performance Metrics RFC 2330 [1]. This document is
parallel to A One-way Delay Metric for IPPM RFC 2679[2]. A sample parallel to A One-way Delay Metric for IPPM RFC 2679[2]. A sample
metric is described that is suitable for application-level measurement metric is described that is suitable for application-level measurement
for streaming multimedia over IP. Using such a measurement, for streaming multimedia over IP. Using such a measurement,
transmission service of a network is probed with a traffic stream transmission service of a network is probed with a traffic stream
similar to that of the application of interest, which is likely to be similar to that of the application of interest, which is likely to be
very dissimilar to the Poisson inter-arrival interval described in [2]. very dissimilar to the Poisson inter-arrival interval described in [2].
Application-level network performance measurement
3. Introduction 3. Introduction
This document discusses concepts relevant to application-level This document discusses concepts relevant to application-level
performance measurements of an IP network. The original driver for performance measurements of an IP network. The original driver for
this work is Quality of Service of interactive periodic streams such this work is Quality of Service of interactive periodic streams such
as multimedia conference over IP, but the idea of application-level as multimedia conference over IP, but the idea of application-level
measurement may have a wider scope. In the following, interactive measurement may have a wider scope. In the following, interactive
multimedia traffic is used as an example to illustrate the concept. multimedia traffic is used as an example to illustrate the concept.
A streaming (hereinafter called periodic) multimedia bit stream may A streaming (hereinafter called periodic) multimedia bit stream may
skipping to change at page 2, line 33 skipping to change at page 2, line 31
regularly spaced singleton tests has some limitations when regularly spaced singleton tests has some limitations when
considered from a general measurement point of view: only part of considered from a general measurement point of view: only part of
the network performance spectrum is sampled. However, from the point the network performance spectrum is sampled. However, from the point
of view of application-level performance, this is actually good news of view of application-level performance, this is actually good news
as explained below. as explained below.
IP delivery service measurements have been discussed within the IP delivery service measurements have been discussed within the
International Telecommunications Union (ITU). A framework for IP International Telecommunications Union (ITU). A framework for IP
service level measurements (with references to the framework for IP service level measurements (with references to the framework for IP
performance [1]) that is intended to be suitable for service planning performance [1]) that is intended to be suitable for service planning
has been approved as I.380[3]. The emphasis in the ITU recommendation has been approved as I.380 [3]. The emphasis in the ITU
is on passive measurements, though not explicitly forbidding active recommendation is on passive measurements, though not explicitly
measurements. The present contribution proposes a method that is forbidding active measurements. The present contribution proposes a
usable both for service planning and end-user testing purposes, method that is usable both for service planning and end-user testing
and is based on active measurements. purposes, and is based on active measurements.
3.1 Terminology 3.1 Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC 2119 [5]. document are to be interpreted as described in RFC 2119 [5].
Although RFC 2119 was written with protocols in mind, the key words Although RFC 2119 was written with protocols in mind, the key words
are used in this document for similar reasons. They are used to are used in this document for similar reasons. They are used to
ensure the results of measurements from two different implementations ensure the results of measurements from two different implementations
are comparable, and to note instances when an implementation could are comparable, and to note instances when an implementation could
skipping to change at page 3, line 27 skipping to change at page 3, line 17
probably true irrespective of the possible QoS mechanism utilized in probably true irrespective of the possible QoS mechanism utilized in
the core network. As an example, for a QoS mechanism without hard the core network. As an example, for a QoS mechanism without hard
guarantees, measurements may be used to ascertain that the "best" guarantees, measurements may be used to ascertain that the "best"
class gets the service that has been promised for the traffic class class gets the service that has been promised for the traffic class
in question. Moreover, an operator could study the quality of a in question. Moreover, an operator could study the quality of a
cheap, low-guarantee service implemented using possible slack cheap, low-guarantee service implemented using possible slack
bandwidth in other classes. Such measurements could be made either bandwidth in other classes. Such measurements could be made either
in studying the feasibility of a new service, or on a regular in studying the feasibility of a new service, or on a regular
basis. basis.
3.3 Measurement types 3.3 Protocol level issues
Delay measurements can be one-way [2,3], paired one-way, or
round-trip [4].
In general, the results of all measurement types may be influenced In general, the results of measurements may be influenced by
by individual application requirements/responses related to the individual application requirements/responses related to the
following issues: following issues:
+ Lost packets: Applications may have varying tolerance to lost + Lost packets: Applications may have varying tolerance to lost
packets. Another consideration is the distribution of lost packets. Another consideration is the distribution of lost
packets (i.e. random or bursty). packets (i.e. random or bursty).
+ Long delays: Many applications will consider packets delayed + Long delays: Many applications will consider packets delayed
longer than a certain value to be equivalent to lost packets longer than a certain value to be equivalent to lost packets
(i.e. real time applications). (i.e. real time applications).
+ Duplicate packets: Some applications may be perturbed if + Duplicate packets: Some applications may be perturbed if
duplicate packets are received. duplicate packets are received.
skipping to change at page 4, line 38 skipping to change at page 4, line 20
identify the sequence of packets received at Dst. identify the sequence of packets received at Dst.
3.4 Application-level measurement 3.4 Application-level measurement
In what follows, a metric is proposed for application-level network In what follows, a metric is proposed for application-level network
performance measurement. In effect, the metric is an emulation of performance measurement. In effect, the metric is an emulation of
periodic multimedia stream performance. The justification for using periodic multimedia stream performance. The justification for using
realistic application metrics in the measurement: realistic application metrics in the measurement:
+ The results of the measurement are automatically relevant to the + The results of the measurement are automatically relevant to the
performance as perceived by application in question. performance as perceived by the application in question.
+ All the packets in the measurement contribute to accuracy of the + All the packets in the measurement contribute to accuracy of the
estimation of performance variation at timescale that is estimation of performance variation at timescale that is
important to the multimedia application (packetization important to the multimedia application (packetization
interval). interval).
+ Effects of elastic traffic (TCP) on measurement packets are + Effects of elastic traffic (TCP) on measurement packets are
different for a sustained stream than for single packets during different for a sustained stream than for single packets during
overloading situations as discussed in [3]. overloading situations as discussed in [3].
3.5 Measurement types 3.5 Measurement types
The measurement may be performed either with synchronized or Delay measurements can be one-way [2,3], paired one-way, or
unsynchronized Src/Dst host clocks. Different possibilities are round-trip [4]. Accordingly, the measurements may be performed
listed below. either with synchronized or unsynchronized Src/Dst host clocks.
Different possibilities are listed below.
3.5.1 One way measurement The reference measurement setup for all measurement types is
shown in Fig. 1.
In the interests of specifying metrics that are as generally usable ----------------< IP >--------------------
as possible, application-level measurements based on one-way delays | | | |
are used in the example metrics. Below, a single one-way measurement ------- ------- -------- --------
is used to keep the example understandable. The implication of | Src | | MP | | MP | | Dst |
application-level measurement for bidirectional applications such as ------- |(Src)| |(Dst) | --------
interactive multimedia conferencing is discussed below. ------- --------
Fig. 1: Example setup for the metric usage.
An example of the use of the metric is a setup with a source host An example of the use of the metric is a setup with a source host
(Src), a destination host (Dst), and corresponding measurement (Src), a destination host (Dst), and corresponding measurement
points (MP(Src) and MP(Dst)) as shown in Figure 1. Separate equipment points (MP(Src) and MP(Dst)) as shown in Figure 1. Separate equipment
for measurement points may be used if having Src and/or Dst conduct for measurement points may be used if having Src and/or Dst conduct
the measurement may significantly affect the delay performance to be the measurement may significantly affect the delay performance to be
measured. MP(Src)should be placed/measured close to the egress point measured. MP(Src)should be placed/measured close to the egress point
of packets from Src. MP(Dst) should be placed/measure close to of packets from Src. MP(Dst) should be placed/measure close to
the ingress point of packets for Dst. "Close" is defined as a the ingress point of packets for Dst. "Close" is defined as a
distance sufficiently small so that application-level performance, distance sufficiently small so that application-level performance
such as delay can be expected to follow the corresponding characteristics measured (such as delay) can be expected to follow
performance characteristic between Src and Dst to an adequate the corresponding performance characteristic between Src and Dst to
accuracy. an adequate accuracy.
----------------< IP >--------------------
| | | |
------- ------- -------- --------
| Src | | MP | | MP | | Dst |
------- |(Src)| |(Dst) | --------
------- --------
Fig. 1: Example setup for the metric usage.
The test setup just described fulfills two important criteria: The test setup just described fulfills two important criteria:
1) Test is made with realistic stream metrics, emulating - for example - 1) Test is made with realistic stream metrics, emulating - for example -
a full-duplex VoIP call. a full-duplex VoIP call.
2) Either one-way or round-trip characteristics may be obtained. 2) Either one-way or round-trip characteristics may be obtained.
It is also possible to have intermediate measurement points between It is also possible to have intermediate measurement points between
MP(Src) and MP(Dst), but that is beyond the scope of this document. MP(Src) and MP(Dst), but that is beyond the scope of this document.
From the viewpoint of a full-duplex VoIP call, the application-level 3.5.1 One way measurement
aspect of measurement may be enhanced by performing two simultaneous
measurements in different directions, as explained below. The extent In the interests of specifying metrics that are as generally usable
to which this is significant depends on issues such as link level as possible, application-level measurements based on one-way delays
technology used. are used in the example metrics. The implication of application-level
measurement for bi-directional applications such as interactive
multimedia conferencing is discussed below.
Performing a single one-way measurement only yields information on
network behavior in one direction. Moreover, the stream at the
network transport level does not emulate accurately a full-duplex
multimedia connection.
3.5.2 Paired one way measurement 3.5.2 Paired one way measurement
Paired one way delay refers to two multimedia streams: Src to Dst Paired one way delay refers to two multimedia streams: Src to Dst
and Dst to Src for the same Src and Dst. By way of example, for and Dst to Src for the same Src and Dst. By way of example, for
some applications, the delay performance of each one way path is some applications, the delay performance of each one way path is
more important than the round trip delay. This is the case for more important than the round trip delay. This is the case for
delay-limited signals such as voice over IP. Possible reasons for delay-limited signals such as voice over IP (VoIP). Possible reasons
difference between one-way delays is different routing of streams for the difference between one-way delays is different routing of
from Src to Dst vs. Dst to Src. streams from Src to Dst vs. Dst to Src.
Moreover, paired one way delay measurement emulates a full-duplex Moreover, paired one way delay measurement emulates a full-duplex
VoIP call more accurately than a single one-way measurement only. VoIP call more accurately than a single one-way measurement only.
3.5.3 Round trip measurement 3.5.3 Round trip measurement
From the point of view of periodic multimedia streams, From the point of view of periodic multimedia streams,
round-trip measurements have two advantages: they avoid the need of round-trip measurements have two advantages: they avoid the need of
host clock synchronization and they allow for a simulation of host clock synchronization and they allow for a simulation of
full-duplex connections. The former aspect means that a measurement full-duplex connections. The former aspect means that a measurement
skipping to change at page 6, line 45 skipping to change at page 6, line 32
the performance estimates thus obtained are pessimistic ones. The the performance estimates thus obtained are pessimistic ones. The
possibility of asymmetric routing and queuing must be taken into possibility of asymmetric routing and queuing must be taken into
account during analysis of the results. account during analysis of the results.
Please note that with suitable arrangements, round-trip measurements Please note that with suitable arrangements, round-trip measurements
may be performed using paired one way measurements. may be performed using paired one way measurements.
4 Sample metric for multimedia stream simulation 4 Sample metric for multimedia stream simulation
The sample metric presented here is similar to the sample metric The sample metric presented here is similar to the sample metric
Type-P-One-way-Delay-Poisson-Stream presented in [2]. Type-P-One-way-Delay-Poisson-Stream presented in [2]. "Singletons", as
defined in [1] and [2] are not directly used in this document because
certain key results (such as duplicate or out of sequence packets)
cannot be identified in the context of a singleton, but only as part
of a sample.
4.1 Metric name 4.1 Metric name
Type-P-One-way-Delay-Periodic-Stream Type-P-One-way-Delay-Periodic-Stream
4.2 Metric parameters 4.2 Metric parameters
4.2.1 Global metric parameters 4.2.1 Global metric parameters
These parameters are applicable to the metrics collected in the
following sections (4.2.2, 4.2.3, and 4.2.4).
+ Src, the IP address of a host + Src, the IP address of a host
+ Dst, the IP address of a host + Dst, the IP address of a host
+ T0, a time + T0, a time, for starting to generate packets and taking
+ Tf, a time, greater than T0 measurements for a sample
+ Tf, a time, greater than T0, for stopping generation of packets
for a sample
+ periodic packet interval incT, a time duration + periodic packet interval incT, a time duration
+ packet size p(j), the number of bytes in each packet of Type-P of + packet size p(j), the number of bytes in each packet of Type-P of
size j size j
+ Tcons, a time interval + dTloss, a time interval, used for determining if a packet should
+ dTloss, a time interval (optional) be considered lost
+ Tcons, a time interval [optional]
While a number of applications will use one packet size (j = 1), While a number of applications will use one packet size (j = 1),
other applications may use packets of different sizes (j > 1). other applications may use packets of different sizes (j > 1).
Especially in cases of congestion, it may be useful to have Especially in cases of congestion, it may be useful to have
packets smaller than the maximum or predominant size of packets packets smaller than the maximum or predominant size of packets
in the periodic stream. in the periodic stream.
4.2.2 Metrics collected at MP(Src) 4.2.2 Metrics collected at MP(Src)
+ Tstamp(Src)[i], for each packet [i], the time of the packet as + Tstamp(Src)[i], for each packet [i], the time of the packet as
measured at MP(Src) measured at MP(Src)
+ PktID [i], for each packet [i], an identification number for the + PktID [i], for each packet [i], an identification number for the
the packet sent from Src to Dst the packet sent from Src to Dst
+ PktSiTy [i], for each packet [i], the packet size and/or type. + PktSiTy [i], for each packet [i], the packet size and/or type.
Some applications may use packets of different size, either Some applications may use packets of different size, either
because of application requirements or in response to IP because of application requirements or in response to IP
performance experienced. performance experienced.
4.2.3 Metrics collected at MP (Dst) 4.2.3 Metrics collected at MP (Dst)
+ dTstop, a time interval, used to add to time Tf to determine when to
stop collecting metrics for a sample
+ Tstamp(Dst)[i], for each packet [i], the time of the packet as + Tstamp(Dst)[i], for each packet [i], the time of the packet as
measured at MP(Dst) measured at MP(Dst)
+ PktID [i], for each packet [i], an identification number for the + PktID [i], for each packet [i], an identification number for the
the packet received at Dst from Src. This identification number the packet received at Dst from Src. This identification number
may be corrupted. may be corrupted.
+ PktSiTy [i], for each packet [i], the packet size and/or type. + PktSiTy [i], for each packet [i], the packet size and/or type.
Some applications may use packets of different size, either Some applications may use packets of different size, either
because of application requirements or in response to IP because of application requirements or in response to IP
performance experienced. performance experienced.
+ PktStatus [i], for each packet [i], the status of the packet + PktStatus [i], for each packet [i], the status of the packet
received. Possible status includes: OK, packet header corrupt, received. Possible status includes: OK, packet header corrupt,
packet payload corrupt, spurious, duplicate packet payload corrupt, spurious, duplicate
4.2.4 Metrics resulting when metrics collected at MP(Src) and MP(Dst) 4.2.4 Metrics resulting when metrics collected at MP(Src) and MP(Dst)
are merged are merged
These parameters are only available as a complete set when the
parameters from the preceding sections (4.2.1, 4.2.2, and 4.2.3 are
combined.
+ Tstamp(Src)[i], for each packet [i], the time of the packet as + Tstamp(Src)[i], for each packet [i], the time of the packet as
measured at MP(Src). This entry may be blank or noted as N/A measured at MP(Src). This entry may be blank or noted as N/A
for spurious packets received at MP(Dst) for spurious packets received at MP(Dst)
+ Tstamp(Dst)[i], for each packet [i], the time of the packet as + Tstamp(Dst)[i], for each packet [i], the time of the packet as
measured at MP(Dst). This entry may be blank or noted as N/A measured at MP(Dst). This entry may be blank or noted as N/A
for packets not received at MP(Dst), received with corrupt for packets not received at MP(Dst), received with corrupt
packet headers, or for duplicate packets received at MP(Dst). packet headers, or for duplicate packets received at MP(Dst).
+ PktID [i], for each packet [i], an identification number for the + PktID [i], for each packet [i], an identification number for the
the packet received. This identification number may be corrupted the packet received. This identification number may be corrupted
for certain packets received at MP (Dst). for certain packets received at MP (Dst).
+ PktSiTy [i], for each packet [i], the packet size and/or type. + PktSiTy [i], for each packet [i], the packet size and/or type.
+ PktStatus [i], for each packet [i], the status of the packet + PktStatus [i], for each packet [i], the status of the packet
received. Possible status includes: OK, packet header corrupt, received. Possible status includes: OK, packet header corrupt,
packet payload corrupt, spurious, duplicate, out of sequence. packet payload corrupt, spurious, duplicate, out of sequence.
+ Delay [i], for each packet [i], the time interval Tstamp(Dst)[i] - + Delay [i], for each packet [i], the time interval Tstamp(Dst)[i] -
Tstamp(Src)[i]. For the following conditions, it will not be Tstamp(Src)[i]. For the following conditions, it will not be
possible to be able to compute delay: possible to be able to compute delay:
skipping to change at page 8, line 30 skipping to change at page 8, line 20
received. Possible status includes: OK, packet header corrupt, received. Possible status includes: OK, packet header corrupt,
packet payload corrupt, spurious, duplicate, out of sequence. packet payload corrupt, spurious, duplicate, out of sequence.
+ Delay [i], for each packet [i], the time interval Tstamp(Dst)[i] - + Delay [i], for each packet [i], the time interval Tstamp(Dst)[i] -
Tstamp(Src)[i]. For the following conditions, it will not be Tstamp(Src)[i]. For the following conditions, it will not be
possible to be able to compute delay: possible to be able to compute delay:
Spurious: There will be no Tstamp(Src)[i] time Spurious: There will be no Tstamp(Src)[i] time
Not received: There will be no Tstamp (Dst) [i] Not received: There will be no Tstamp (Dst) [i]
Corrupt packet header: There will be no Tstamp (Dst) [i] Corrupt packet header: There will be no Tstamp (Dst) [i]
Duplicate: Only the first non-corrupt copy of the packet Duplicate: Only the first non-corrupt copy of the packet
received at Dst should have Delay [i] computed. received at Dst should have Delay [i] computed.
+ DJit[i], for each packet [i] except the first one: momentary + SDV[i] [optional] , for each packet [i] except the first one:
delay variation, i.e., the time interval Tstamp(Dst)[i]- momentary delay variation between successive packets, i.e., the
Tstamp(Dst)[i-1] - (Tstamp(Src)[i]-Tstamp(Src)[i-1]. time interval Delay[i] - Delay [i-1]. SDV[i] may be negative,
Applicability of jitter: delay must be calculable for both zero, or positive. Delay for both packets i and i+1 must be
packets i and i+1 according to the definition above. calculable according to the definition above or SDV[i] is
undefined.
4.4 Definition 4.3 High level description of the procedure to collect a sample
Beginning on or after time T0, Type-P packets are generated Beginning on or after time T0, Type-P packets are generated
by Src and sent to Dst until time Tf is reached with a nominal by Src and sent to Dst until time Tf is reached with a nominal
interval between packets of incT. interval between the first bit of successive packets of incT as
measured at MP(Src). incT may be nominal due to a number of reasons:
variation in packet generation at Src, clock issues (see section 4.7),
etc.
MP(Src) records the following information only for packets with MP(Src) records the following information only for packets with
timestamps between and including T0 and Tf: timestamp, packet timestamps between and including T0 and Tf: timestamp,
identifier, and packet size/type of each packet sent from Src to packet identifier, and packet size/type of each packet sent from Src
Dst that is part of the sample. to Dst that is part of the sample.
MP (Dst) records the following information only for packets with MP (Dst) records the following information only for packets with
time stamps between T0 and Tf: timestamp, packet identifier, time stamps between T0 and (Tf+ dTstop): timestamp, packet identifier,
packet size/type, and received status of each packet received from packet size/type, and received status of each packet received from
Src at Dst that is part of the sample. At a time Tcons > Tf, the Src at Dst that is part of the sample. Optionally, at a time Tf +
data from MP(Src) and MP(Dst) are consolidated to derive the Tcons, the data from MP(Src) and MP(Dst) are consolidated to derive
results of the sample metric. the results of the sample metric.
4.5 Discussion To prevent stopping data collection too soon, dTcons should be greater
than or equal to dTstop. Conversely, to keep data collection
reasonably efficient, dTstop should be some reasonable time interval
(seconds/minutes/hours), even if dTloss is infinite or extremely long.
4.4 Discussion
The sample metric thus defined is intended to probe the delays and The sample metric thus defined is intended to probe the delays and
the delay variation as experienced by multimedia streams of the delay variation as experienced by multimedia streams of
an application. Subsequently, the delay is assumed to be measured at an application. Subsequently, the delay is assumed to be measured at
transport layer level. Since a range of packet sizes and nominal transport layer level. Since a range of packet sizes and nominal
interval between packets is used, the method probes only a specific interval between packets is used, the method probes only a specific
time scale of network QoS variations. time scale of network QoS variations.
There are a number of factors that should be taken into account when There are a number of factors that should be taken into account when
collecting a sample metric of Type-P-One-way-Delay-Periodic-Stream. collecting a sample metric of Type-P-One-way-Delay-Periodic-Stream.
+ T0 and Tf should specify a long enough time interval to + T0 and (Tf + dTloss) should specify a long enough time interval to
represent a reasonable use of the application under test (e.g. do represent a reasonable use of the application under test (e.g. do
not provide only a 100 ms time interval for a phone call) not provide only a 100 ms time interval for a phone call)
+ T0 and Tf should specify a time interval that is not excessively + T0 and (Tf + dTloss) should specify a time interval that is not
long compared to the usage of the application under (e.g. do not excessively long compared to the usage of the application under test
provide a one week continuous phone call) (e.g. do not provide a one week continuous phone call)
+ The nominal interval between packets (incT) and the packet size(s) + The nominal interval between packets (incT) and the packet size(s)
(p(j)) should not define an equivalent bit rate that is in excess (p(j)) should not define an equivalent bit rate that is in excess
of the capacity of the egress port of Src, the ingress port of Dst, of the capacity of the egress port of Src, the ingress port of Dst,
or the carrying capacity of the intervening network(s). There may or the carrying capacity of the intervening network(s). There may
be exceptional cases to test the response of the application to be exceptional cases to test the response of the application to
overload conditions in the transport networks, but these cases overload conditions in the transport networks, but these cases
should be strictly controlled. should be strictly controlled.
+ Real delay values will be positive. Therefore, it does not make + Real delay values will be positive. Therefore, it does not make
skipping to change at page 10, line 7 skipping to change at page 9, line 52
synchronization to within several 10s of usec. Ordinary application synchronization to within several 10s of usec. Ordinary application
of NTP may allow synchronization to within several msec, but this of NTP may allow synchronization to within several msec, but this
depends on the stability and symmetry of delay properties among those depends on the stability and symmetry of delay properties among those
NTP agents used, and this delay is what we are trying to measure. A NTP agents used, and this delay is what we are trying to measure. A
combination of some GPS-based NTP servers and a conservatively combination of some GPS-based NTP servers and a conservatively
designed and deployed set of other NTP servers should yield good designed and deployed set of other NTP servers should yield good
results, but this is yet to be tested. results, but this is yet to be tested.
+ A given methodology will have to include a way to determine + A given methodology will have to include a way to determine
whether a delay value is infinite or whether it is merely very whether a delay value is infinite or whether it is merely very
large (and the packet is yet to arrive at Dst). This may be achieved large (and the packet is yet to arrive at Dst). The global metric
by using the optional global metric parameter dTloss, which defines parameter dTloss defines a time interval such that delays larger
a time interval such that delays larger than dTloss are interpreted than dTloss are interpreted as losses.
as losses.
{Comment: Note that, for many applications of these metrics, the {Comment: Note that, for many applications of these metrics, the
harm in treating a large delay as infinite might be zero or very harm in treating a large delay as infinite might be zero or very
small. A TCP data packet, for example, that arrives only after small. A TCP data packet, for example, that arrives only after
several multiples of the RTT may as well have been lost.} several multiples of the RTT may as well have been lost.}
4.6 Additional Methodology Aspects 4.5 Additional Methodology Aspects
As with other Type-P-* metrics, the detailed methodology will depend As with other Type-P-* metrics, the detailed methodology will depend
on the Type-P (e.g., protocol number, UDP/TCP port number, size, on the Type-P (e.g., protocol number, UDP/TCP port number, size,
precedence). precedence).
4.7 Errors and uncertainties 4.6 Errors and uncertainties
The description of any specific measurement method should include an The description of any specific measurement method should include an
accounting and analysis of various sources of error or uncertainty. accounting and analysis of various sources of error or uncertainty.
The Framework document [1] provides general guidance on this point, The Framework document [1] provides general guidance on this point,
but we note here the following specifics related to delay metrics: but we note here the following specifics related to delay metrics:
+ Errors or uncertainties due to uncertainties in the clocks of the + Errors or uncertainties due to uncertainties in the clocks of the
MP(Src) and MP(Dst) measurement points. MP(Src) and MP(Dst) measurement points.
+ Errors or uncertainties due to the difference between 'wire time' + Errors or uncertainties due to the difference between 'wire time'
and 'host time'. and 'host time'.
4.7.1. Errors or uncertainties related to Clocks 4.6.1. Errors or uncertainties related to Clocks
The uncertainty in a measurement of one-way delay is related, in The uncertainty in a measurement of one-way delay is related, in
part, to uncertainties in the clocks of MP(Src) and MP(Dst). In part, to uncertainties in the clocks of MP(Src) and MP(Dst). In
the following, we refer to the clock used to measure when the packet the following, we refer to the clock used to measure when the packet
was measured at MP(Src) as the MP(Src) clock and we refer to the was measured at MP(Src) as the MP(Src) clock and we refer to the
clock used to measure when the packet was received at MP(Dst) as the clock used to measure when the packet was received at MP(Dst) as the
MP(Dst) clock. Alluding to the notions of synchronization, accuracy, MP(Dst) clock. Alluding to the notions of synchronization, accuracy,
resolution, and skew, we note the following: resolution, and skew, we note the following:
+ Any error in the synchronization between the MP(Src) clock and + Any error in the synchronization between the MP(Src) clock and
skipping to change at page 11, line 38 skipping to change at page 11, line 25
synchronization. Thus, |Tsynch(t)| <= Esynch(t). synchronization. Thus, |Tsynch(t)| <= Esynch(t).
Taking these items together, we note that naive computation Taking these items together, we note that naive computation
Tstamp(Dst)[i] - Tstamp(Src) [i] will be off by Tsynch(t) +/- Tstamp(Dst)[i] - Tstamp(Src) [i] will be off by Tsynch(t) +/-
(ResMP(SRc) + ResMP(Dst)). Using the notion of Esynch(t), we note (ResMP(SRc) + ResMP(Dst)). Using the notion of Esynch(t), we note
that these clock-related problems introduce a total uncertainty of that these clock-related problems introduce a total uncertainty of
Esynch(t)+ Rsource + Rdest. This estimate of total clock-related Esynch(t)+ Rsource + Rdest. This estimate of total clock-related
uncertainty should be included in the error/uncertainty analysis of uncertainty should be included in the error/uncertainty analysis of
any measurement implementation. any measurement implementation.
4.7.2. Errors or uncertainties related to Wire-time vs Host-time 4.6.2. Errors or uncertainties related to Wire-time vs Host-time
As we have defined one-way periodic delay, we would like to measure As we have defined one-way periodic delay, we would like to measure
the time between when a packet is measured and time-stamped at the time between when a packet is measured and time-stamped at
MP(Src) and when it arrives and is time-stamped at MP(Dst) and we MP(Src) and when it arrives and is time-stamped at MP(Dst) and we
refer to these as "wire times." If the timings are themselves refer to these as "wire times." If the timings are themselves
performed by software on Src and Dst, however, then this software can performed by software on Src and Dst, however, then this software can
only directly measure the time between when Src generates the packet only directly measure the time between when Src generates the packet
just prior to sending the test packet and when Dst has started to just prior to sending the test packet and when Dst has started to
process the packet after having received the test packet, and we refer process the packet after having received the test packet, and we refer
to these two points as "host times". to these two points as "host times".
skipping to change at page 12, line 16 skipping to change at page 11, line 53
host time is uncertain, this uncertainty must be accounted for in an host time is uncertain, this uncertainty must be accounted for in an
analysis of a given measurement method. We denote by Hsource an analysis of a given measurement method. We denote by Hsource an
upper bound on the uncertainty in the difference between wire time upper bound on the uncertainty in the difference between wire time
of MP(Src) and host time on the Src host, and similarly define Hdest of MP(Src) and host time on the Src host, and similarly define Hdest
for the difference between the host time on the Dst host and the wire for the difference between the host time on the Dst host and the wire
time of MP(Dst). We then note that these problems introduce a total time of MP(Dst). We then note that these problems introduce a total
uncertainty of Hsource+Hdest. This estimate of total wire-vs-host uncertainty of Hsource+Hdest. This estimate of total wire-vs-host
uncertainty should be included in the error/uncertainty analysis of uncertainty should be included in the error/uncertainty analysis of
any measurement implementation. any measurement implementation.
4.7.3. Calibration 4.6.3. Calibration
Generally, the measured values can be decomposed as follows: Generally, the measured values can be decomposed as follows:
measured value = true value + systematic error + random error measured value = true value + systematic error + random error
If the systematic error (the constant bias in measured values) can be If the systematic error (the constant bias in measured values) can be
determined, it can be compensated for in the reported results. determined, it can be compensated for in the reported results.
reported value = measured value - systematic error reported value = measured value - systematic error
therefore therefore
reported value = true value + random error reported value = true value + random error
The goal of calibration is to determine the systematic and random The goal of calibration is to determine the systematic and random
skipping to change at page 14, line 5 skipping to change at page 13, line 34
recording the measurements), all of which may increase the random recording the measurements), all of which may increase the random
error in measured singletons. Therefore, in addition to minimal load error in measured singletons. Therefore, in addition to minimal load
measurements to find the systematic error, calibration measurements measurements to find the systematic error, calibration measurements
should be performed with the same measurement load that the should be performed with the same measurement load that the
instruments will see in the field. instruments will see in the field.
We wish to reiterate that this statistical treatment refers to the We wish to reiterate that this statistical treatment refers to the
calibration of the instrument; it is used to "calibrate the meter calibration of the instrument; it is used to "calibrate the meter
stick" and say how well the meter stick reflects reality. stick" and say how well the meter stick reflects reality.
4.8 Reporting the metric 4.7 Reporting the metric
The calibration and context in which the metric is measured MUST be The calibration and context in which the metric is measured MUST be
carefully considered, and SHOULD always be reported along with metric carefully considered, and SHOULD always be reported along with metric
results. We now present four items to consider: the Type-P of test results. We now present five items to consider: the Type-P of test
packets, the threshold of delay equivalent to loss (if any), error packets, the threshold of delay equivalent to loss, error
calibration, and the path traversed by the test packets. This list calibration, the path traversed by the test packets, and background
is not exhaustive; any additional information that could be useful conditions at Src, Dst, and the intervening networks during a sample.
in interpreting applications of the metrics should also be reported. This list is not exhaustive; any additional information that could be
useful in interpreting applications of the metrics should also be
reported.
4.8.1. Type-P 4.7.1. Type-P
As noted in the Framework document [1], the value of the metric may As noted in the Framework document [1], the value of the metric may
depend on the type of IP packets used to make the measurement, or depend on the type of IP packets used to make the measurement, or
"type-P". The value of Type-P-One-way-Periodic-Delay could change "type-P". The value of Type-P-One-way-Periodic-Delay could change
if the protocol (UDP or TCP), port number, size, or arrangement for if the protocol (UDP or TCP), port number, size, or arrangement for
special treatment (e.g., IP precedence or RSVP) changes. The exact special treatment (e.g., IP precedence or RSVP) changes. The exact
Type-P used to make the measurements MUST be accurately reported. Type-P used to make the measurements MUST be accurately reported.
4.8.2. Threshold for delay equivalent to loss 4.7.2. Threshold for delay equivalent to loss
In addition, the threshold for delay equivalent to loss (or In addition, the threshold for delay equivalent to loss (or
methodology to determine this threshold) MUST be reported. methodology to determine this threshold) MUST be reported.
4.8.3. Calibration results 4.7.3. Calibration results
+ If the systematic error can be determined, it SHOULD be removed + If the systematic error can be determined, it SHOULD be removed
from the measured values. from the measured values.
+ You SHOULD also report the calibration error, e, such that the + You SHOULD also report the calibration error, e, such that the
true value is the reported value plus or minus e, with 95% true value is the reported value plus or minus e, with 95%
confidence (see the last section.) confidence (see the last section.)
+ If possible, the conditions under which a test packet with finite + If possible, the conditions under which a test packet with finite
delay is reported as lost due to resource exhaustion on the delay is reported as lost due to resource exhaustion on the
measurement instrument SHOULD be reported. measurement instrument SHOULD be reported.
4.8.4. Path 4.7.4. Path
Finally, the path traversed by the packets SHOULD be reported, if Finally, the path traversed by the packets SHOULD be reported, if
possible. In general it is impractical to know the precise path a possible. In general it is impractical to know the precise path a
given packet takes through the network. The precise path may be given packet takes through the network. The precise path may be
known for certain Type-P packets on short or stable paths. If known for certain Type-P packets on short or stable paths. If
Type-P includes the record route (or loose-source route) option in Type-P includes the record route (or loose-source route) option in
the IP header, and the path is short enough, and all routers* on the the IP header, and the path is short enough, and all routers* on the
path support record (or loose-source) route, then the path will be path support record (or loose-source) route, then the path will be
precisely recorded. precisely recorded.
skipping to change at page 15, line 16 skipping to change at page 14, line 40
many routers do not support (or are not configured for) record route, many routers do not support (or are not configured for) record route,
and use of this feature would often artificially worsen the and use of this feature would often artificially worsen the
performance observed by removing the packet from common-case performance observed by removing the packet from common-case
processing. However, partial information is still valuable context. processing. However, partial information is still valuable context.
For example, if a host can choose between two links* (and hence two For example, if a host can choose between two links* (and hence two
separate routes from Src to Dst), then the initial link used is separate routes from Src to Dst), then the initial link used is
valuable context. {Comment: For example, with Merit's NetNow setup, valuable context. {Comment: For example, with Merit's NetNow setup,
a Src on one NAP can reach a Dst on another NAP by either of several a Src on one NAP can reach a Dst on another NAP by either of several
different backbone networks.} different backbone networks.}
4.7.5 Background conditions
In many cases, the results of a sample may be influenced by conditions
at Src, Dst, and/or any intervening networks. Some things that may
affect the results of a sample include: traffic levels and/or bursts
during the sample, link and/or host failures, etc. Information about
the background conditions may only be available by non-Internet means
(e.g. phone calls, television) and may only become available days after
samples are taken.
4.8 Single sample vs. a "sample of samples"
Because this metric represents a periodic stream as one sample, there
may be value in running multiple tests using this metric to collect
a "sample of samples". For example, it may be more appropriate to
test 1,000 two-minute VoIP calls rather than a single 2,000 minute
VoIP call. When considering collection of a sample of samples, issues
like the interval between samples (e.g. Poisson vs. periodic, time of
day/day of week), composition of samples (e.g. equal (Tf-T0 duration,
different packet sizes), and network considerations (e.g. run different
samples over different intervening link-host combinations) should be
taken into account. For items like the interval between samples,
the pattern of use of the application being measured should be
considered.
4.9 Statistics based on Type-P-One-way-Delay-Periodic-Stream 4.9 Statistics based on Type-P-One-way-Delay-Periodic-Stream
4.9.1 Statistics calculable from one sample
As a metric based on a sample representative of certain As a metric based on a sample representative of certain
applications, some normal statistics such as median and percentile applications, some general purpose statistics (e.g. median and
may be less applicable than ways to characterize the range of delay percentile) may be less applicable than ways to characterize the
values recorded during the sample metrics. range of delay values recorded during the sample metrics.
Example, a sample metric generates 100 packets as measured at MP(Src) Example, a sample metric generates 100 packets as measured at MP(Src)
with the following measurements at MP(Dst) with the following measurements at MP(Dst)
+ 80 packets received with delay [i] <= 20 ms + 80 packets received with delay [i] <= 20 ms
+ 8 packets received with delay [i] > 20 ms + 8 packets received with delay [i] > 20 ms
+ 5 packets received with corrupt packet headers + 5 packets received with corrupt packet headers
+ 4 packets from MP(Src) with no matching packet recorded + 4 packets from MP(Src) with no matching packet recorded
at MP(Dst) (effectively lost) at MP(Dst) (effectively lost)
+ 3 packets received with corrupt packet payload and + 3 packets received with corrupt packet payload and
and delay [i] <= 20 ms and delay [i] <= 20 ms
+ 2 packets that duplicate one of the 80 packets received + 2 packets that duplicate one of the 80 packets received
correctly in the first line correctly in the first line
For this example, packets are considered acceptable if they are For this example, packets are considered acceptable if they are
received with less than or equal to 20ms delays and without corrupt received with less than or equal to 20ms delays and without corrupt
packet headers or packet payload. In this case, the percentage packet headers or packet payload. In this case, the percentage
of acceptable packets is 80/100 = 80%. of acceptable packets is 80/100 = 80%.
In the case where the application will accept packets with corrupt For a different application which will accept packets with corrupt
packet payload and no delay bound (so long as the packet is received), packet payload and no delay bound (so long as the packet is received),
the percentage of acceptable packets is (80+8+3)/100 = 91%. the percentage of acceptable packets is (80+8+3)/100 = 91%.
4.9.2 Statistics calculable from multiple samples
For computing statistics, a "sample of samples" series of
measurements may be performed. As discussed in section 4.8, under
these conditions, general purpose statistics (e.g. median, percentile,
etc.) may be more relevant as a more statistically significant
number of packets are used.
5. Security Considerations 5. Security Considerations
5.1 Denial of Service Attacks 5.1 Denial of Service Attacks
This metric generates a periodic stream of packets from one host (Src) This metric generates a periodic stream of packets from one host (Src)
to another host (Dst) through intervening networks. This metric to another host (Dst) through intervening networks. This metric
could be abused for denial of service attacks directed at Dst and/or could be abused for denial of service attacks directed at Dst and/or
the intervening network(s). the intervening network(s).
Administrators of Src, Dst, and the intervening network(s) should Administrators of Src, Dst, and the intervening network(s) should
skipping to change at page 16, line 31 skipping to change at page 16, line 40
packets are part of a sample metric. With that knowledge at Dst packets are part of a sample metric. With that knowledge at Dst
and/or the intervening networks, it is possible to change the and/or the intervening networks, it is possible to change the
processing of the packets (e.g. increasing or decreasing delay) processing of the packets (e.g. increasing or decreasing delay)
that may distort the measured performance. It may also be that may distort the measured performance. It may also be
possible to generate additional packets that appear to be part of possible to generate additional packets that appear to be part of
the sample metric. These additional packets are likely to perturb the sample metric. These additional packets are likely to perturb
the results of the sample measurement. the results of the sample measurement.
6. Acknowledgements 6. Acknowledgements
The authors wish to thank the chairs of the IPPM WG for comments The authors wish to thank the chairs of the IPPM WG, Howard
that have made the present draft clearer and more focused. The Stanislevic and Al Morton for comments that have made the present
authors have also built substantially on the foundations laid by the draft clearer and more focused. The authors have also built on the
authors of the framework for IP performance [1]. substantial foundations laid by the authors of the framework for
IP performance [1].
7. References 7. References
[1] V.Paxson, G.Almes, J.Mahdavi, and M.Mathis: Framework for IP [1] V.Paxson, G.Almes, J.Mahdavi, and M.Mathis: Framework for IP
Performance Metrics, IETF RFC 2330, May 1998. Performance Metrics, IETF RFC 2330, May 1998.
[2] G.Almes, S.Kalidindi, and M.Zekauskas: A one-way delay metric [2] G.Almes, S.Kalidindi, and M.Zekauskas: A one-way delay metric
for IPPM, IETF RFC 2679, September 1999. for IPPM, IETF RFC 2679, September 1999.
[3] International Telecommunications Union recommendation I.380, [3] International Telecommunications Union recommendation I.380,
February 1999. February 1999.
[4] G.Almes, S.Kalidindi, and M.Zekauskas: A round-trip delay [4] G.Almes, S.Kalidindi, and M.Zekauskas: A round-trip delay
skipping to change at line 782 skipping to change at line 840
Fax. +358 9 4376 6852 Fax. +358 9 4376 6852
Glenn Grotefeld <g.grotefeld@motorola.com> Glenn Grotefeld <g.grotefeld@motorola.com>
Motorola, Inc. Motorola, Inc.
1303 E. Algonquin Road 1303 E. Algonquin Road
4th Floor 4th Floor
Schaumburg, IL 60196 Schaumburg, IL 60196
USA USA
Phone +1 847 576-5992 Phone +1 847 576-5992
Fax +1 847 538-7455 Fax +1 847 538-7455
EXPIRES NOVEMBER 2000
 End of changes. 

This html diff was produced by rfcdiff 1.23, available from http://www.levkowetz.com/ietf/tools/rfcdiff/