--- 1/draft-ietf-ippm-npmps-03.txt 2006-02-04 23:45:47.000000000 +0100 +++ 2/draft-ietf-ippm-npmps-04.txt 2006-02-04 23:45:47.000000000 +0100 @@ -1,18 +1,18 @@ Network Working Group V. Raisanen INTERNET-DRAFT Nokia -Expiration Date: May 2001 G. Grotefeld +Expiration Date: July 2001 G. Grotefeld Motorola - November 2000 + January 2001 Network performance measurement for periodic streams - + 1. Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. @@ -38,21 +38,21 @@ level IP network transport measurement for periodic streams, such as VoIP or streaming multimedia over IP. In this document, the reader is assumed to be familiar with the terminology of the Framework for IP Performance Metrics RFC 2330 [1]. This document is parallel to A One-way Delay Metric for IPPM RFC 2679 [2]. Although this document is based on the delay metrics, other characteristics can be measured with this approach, too. For example, packet loss rate, reordering / out-of sequence, and successive delay variation are all additional metrics which can be built from this baseline set of measurements. -Raisanen, Grotefeld expires May 2001 [Page 1] +Raisanen, Grotefeld expires July 2001 [Page 1] 3. Introduction This document discusses concepts relevant to application-level performance measurements of an IP network. The original driver for this work is Quality of Service of interactive periodic streams such as multimedia conference over IP, but the idea of application-level measurement may have a wider scope. In the following, interactive multimedia traffic is used as an example to illustrate the concept. @@ -84,21 +84,21 @@ The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [4]. Although RFC 2119 was written with protocols in mind, the key words are used in this document for similar reasons. They are used to ensure the results of measurements from two different implementations are comparable, and to note instances when an implementation could perturb the network. -Raisanen, Grotefeld expires May 2001 [Page 2] +Raisanen, Grotefeld expires July 2001 [Page 2] 3.2 Considerations related to delay For interactive multimedia sessions, end-to-end delay is an important factor. Too large a delay reduces the quality of the multimedia session as perceived by the participants. One approach for managing end-to-end delays on an Internet path involving heterogeneous link layer technologies is to use per-domain delay quotas (e.g. 50 ms for a particular IP domain). The 50 ms would then be included into a calculation of an end-to-end delay bound. A @@ -134,21 +134,21 @@ Fig.1 illustrates measurements on multiple protocol levels that are relevant to this draft. The major focus of the present draft is on transport quality evaluation from application point of view. However, to properly account for quality effects of, e.g., operating system and codec on packet voice, it is beneficial to be able to measure quality at IP level [5]. Link layer monitoring provides a way of accounting for link layer characteristics such as bit error rates. -Raisanen, Grotefeld expires May 2001 [Page 3] +Raisanen, Grotefeld expires July 2001 [Page 3] --------------- | application | --------------- | transport | <-- --------------- | network | <-- --------------- | link | <-- --------------- | physical | @@ -186,21 +186,21 @@ corruption. + Spurious packet: Dst may receive spurious packets (i.e. packets that are not sent by the Src as part of the metric). Many applications may be perturbed by spurious packets. Depending, e.g., on the observed protocol level, some issues listed above may be indistinguishable from others by the application, it may be important to preserve the distinction for the operators of Src, Dst, and/or the intermediate network(s). -Raisanen, Grotefeld expires May 2001 [Page 4] +Raisanen, Grotefeld expires July 2001 [Page 4] Because of the possible errors listed above, in most cases it is recommended to use a packet identifier for each packet generated at Src. Identifiers for the metric sample may be those used by the underlying transport layer (e.g. RTP sequence number) or the same identifiers used by an application if the application to be modeled by the metric uses an identifier. The possibility of identifier roll-over (reuse if intentional) during a metric collected over a "long" (application dependent) time should be observed. If the application does not use an identifier, it may still be @@ -230,21 +230,21 @@ 3.5 Measurement types Delay measurements can be one-way [2,3], paired one-way, or round-trip [6]. Accordingly, the measurements may be performed either with synchronized or unsynchronized Src/Dst host clocks. Different possibilities are listed below. The reference measurement setup for all measurement types is shown in Fig. 2. -Raisanen, Grotefeld expires May 2001 [Page 5] +Raisanen, Grotefeld expires July 2001 [Page 5] ----------------< IP >-------------------- | | | | ------- ------- -------- -------- | Src | | MP | | MP | | Dst | ------- |(Src)| |(Dst) | -------- ------- -------- Fig. 2: Example setup for the metric usage. An example of the use of the metric is a setup with a source host @@ -279,21 +279,21 @@ as possible, application-level measurements based on one-way delays are used in the example metrics. The implication of application-level measurement for bi-directional applications such as interactive multimedia conferencing is discussed below. Performing a single one-way measurement only yields information on network behavior in one direction. Moreover, the stream at the network transport level does not emulate accurately a full-duplex multimedia connection. -Raisanen, Grotefeld expires May 2001 [Page 6] +Raisanen, Grotefeld expires July 2001 [Page 6] 3.5.2 Paired one way measurement Paired one way delay refers to two multimedia streams: Src to Dst and Dst to Src for the same Src and Dst. By way of example, for some applications, the delay performance of each one way path is more important than the round trip delay. This is the case for delay-limited signals such as VoIP. Possible reasons for the difference between one-way delays is different routing of streams from Src to Dst vs. Dst to Src. @@ -326,21 +326,21 @@ original sender may be more bursty than the one on the first "leg" of the round-trip journey. The last issue, however, means in practice that returning stream experiences worse QoS than the other one, and the performance estimates thus obtained are pessimistic ones. The possibility of asymmetric routing and queuing must be taken into account during analysis of the results. Please note that with suitable arrangements, round-trip measurements may be performed using paired one way measurements. -Raisanen, Grotefeld expires May 2001 [Page 7] +Raisanen, Grotefeld expires July 2001 [Page 7] 4 Sample metric for multimedia stream simulation The sample metric presented here is similar to the sample metric Type-P-One-way-Delay-Poisson-Stream presented in [2]. "Singletons", as defined in [1] and [2] are not directly used in this document because certain key results (such as duplicate or out of sequence packets) cannot be identified in the context of a singleton, but only as part of a sample. @@ -355,41 +355,41 @@ These parameters are applicable to the metrics collected in the following sections (4.2.2, 4.2.3, and 4.2.4). + Src, the IP address of a host + Dst, the IP address of a host + IPV, the IP version (IPv4/IPv6) used in the measurement + T0, a time, for starting to generate packets and taking measurements for a sample + Tf, a time, greater than T0, for stopping generation of packets for a sample - + periodic packet interval incT, a time duration + + incT, nominal duration of inter-packet interval + packet size p(j), the number of bytes in each packet of Type-P of size j + dTloss, a time interval, used for determining if a packet should be considered lost + Tcons, a time interval [optional] While a number of applications will use one packet size (j = 1), other applications may use packets of different sizes (j > 1). Especially in cases of congestion, it may be useful to have packets smaller than the maximum or predominant size of packets in the periodic stream. 4.2.2 Metrics collected at MP(Src) + Tstamp(Src)[i], for each packet [i], the time of the packet as measured at MP(Src) + PktID [i], for each packet [i], an identification number for the the packet sent from Src to Dst. -Raisanen, Grotefeld expires May 2001 [Page 8] +Raisanen, Grotefeld expires July 2001 [Page 8] + PktSiTy [i], for each packet [i], the packet size and/or type. Some applications may use packets of different size, either because of application requirements or in response to IP performance experienced. 4.2.3 Metrics collected at MP (Dst) + dTstop, a time interval, used to add to time Tf to determine when to stop collecting metrics for a sample + Tstamp(Dst)[i], for each packet [i], the time of the packet as @@ -419,21 +419,21 @@ for packets not received at MP(Dst), received with corrupt packet headers, or for duplicate packets received at MP(Dst). + PktID [i], for each packet [i], an identification number for the the packet received. This identification number may be corrupted for certain packets received at MP (Dst). + PktSiTy [i], for each packet [i], the packet size and/or type. + PktStatus [i], for each packet [i], the status of the packet received. Possible status includes: OK, packet header corrupt, packet payload corrupt, spurious, duplicate, out-of-sequence. -Raisanen, Grotefeld expires May 2001 [Page 9] +Raisanen, Grotefeld expires July 2001 [Page 9] + Delay [i], for each packet [i], the time interval Tstamp(Dst)[i] - Tstamp(Src)[i]. For the following conditions, it will not be possible to be able to compute delay: Spurious: There will be no Tstamp(Src)[i] time Not received: There will be no Tstamp (Dst) [i] Corrupt packet header: There will be no Tstamp (Dst) [i] Duplicate: Only the first non-corrupt copy of the packet received at Dst should have Delay [i] computed. + SDV[i] [optional] , for each packet [i] except the first one: momentary delay variation between successive packets, i.e., the @@ -461,27 +461,28 @@ packet size/type, and received status of each packet received from Src at Dst that is part of the sample. Optionally, at a time Tf + Tcons, the data from MP(Src) and MP(Dst) are consolidated to derive the results of the sample metric. To prevent stopping data collection too soon, dTcons should be greater than or equal to dTstop. Conversely, to keep data collection reasonably efficient, dTstop should be some reasonable time interval (seconds/minutes/hours), even if dTloss is infinite or extremely long. -Raisanen, Grotefeld expires May 2001 [Page 10] +Raisanen, Grotefeld expires July 2001 [Page 10] 4.4 Discussion The sample metric thus defined is intended to probe the delays and the delay variation as experienced by multimedia streams of - an application. Subsequently, the delay is assumed to be measured at + an application. Due to the definition of the metric, also packet loss + status of packets is recorded. The delay is assumed to be measured at transport layer level. Since a range of packet sizes and nominal interval between packets is used, the method probes only a specific time scale of network QoS variations. There are a number of factors that should be taken into account when collecting a sample metric of Type-P-One-way-Delay-Periodic-Stream. + T0 and (Tf + dTloss) should specify a long enough time interval to represent a reasonable use of the application under test (e.g. do not provide only a 100 ms time interval for a phone call) @@ -512,21 +513,21 @@ depends on the stability and symmetry of delay properties among those NTP agents used, and this delay is what we are trying to measure. A combination of some GPS-based NTP servers and a conservatively designed and deployed set of other NTP servers should yield good results, but this is yet to be tested. + Reordering of packets is best discussed in terms of the entire set of measurement packets received, i.e. should be addressed in Sec. 4.9.1. -Raisanen, Grotefeld expires May 2001 [Page 11] +Raisanen, Grotefeld expires July 2001 [Page 11] + A given methodology will have to include a way to determine whether packet was lost or whether delay is merely very large (and the packet is yet to arrive at Dst). The global metric parameter dTloss defines a time interval such that delays larger than dTloss are interpreted as losses. {Comment: Note that, for many applications of these metrics, the harm in treating a large delay as infinite might be zero or very small. A TCP data packet, for example, that arrives only after several multiples of the RTT may as well have been lost.} @@ -534,25 +535,27 @@ As with other Type-P-* metrics, the detailed methodology will depend on the Type-P (e.g., protocol number, UDP/TCP port number, size, precedence). 4.6 Errors and uncertainties The description of any specific measurement method should include an accounting and analysis of various sources of error or uncertainty. The Framework document [1] provides general guidance on this point, - but we note here the following specifics related to delay metrics: + but we note here the following specifics related to periodic + streams and delay metrics: + + Error due to variation of incT. The reasons for this can be e.g. + uneven process scheduling, possibly due to CPU load. + Errors or uncertainties due to uncertainties in the clocks of the MP(Src) and MP(Dst) measurement points. - + Errors or uncertainties due to the difference between 'wire time' and 'host time'. 4.6.1. Errors or uncertainties related to Clocks The uncertainty in a measurement of one-way delay is related, in part, to uncertainties in the clocks of MP(Src) and MP(Dst). In the following, we refer to the clock used to measure when the packet was measured at MP(Src) as the MP(Src) clock and we refer to the clock used to measure when the packet was received at MP(Dst) as the @@ -561,21 +564,21 @@ + Any error in the synchronization between the MP(Src) clock and the MP(Dst) clock will contribute to error in the delay measurement. We say that the MP(Src) clock and the MP(Dst) clock have a synchronization error of Tsynch if the MP(Src) clock is Tsynch ahead of the MP(Dst) clock. Thus, if we know the value of Tsynch exactly, we could correct for clock synchronization by adding Tsynch to the uncorrected value of Tstamp(Dst)[i] - Tstamp(Src) [i]. -Raisanen, Grotefeld expires May 2001 [Page 12] +Raisanen, Grotefeld expires July 2001 [Page 12] + The accuracy of a clock is important only in identifying the time at which a given delay was measured. Accuracy, per se, has no importance to the accuracy of the measurement of delay. When computing delays, we are interested only in the differences between clock values, not the values themselves. + The resolution of a clock adds to uncertainty about any time measured with it. Thus, if the MP(Src) clock has a resolution of 10 msec, then this adds 10 msec of uncertainty to any time value measured with it. We will denote the resolution of the source @@ -611,21 +614,21 @@ only directly measure the time between when Src generates the packet just prior to sending the test packet and when Dst has started to process the packet after having received the test packet, and we refer to these two points as "host times". To the extent that the difference between wire time and host time is accurately known, this knowledge can be used to correct for wire time measurements and the corrected value more accurately estimates the desired (host time) metric. -Raisanen, Grotefeld expires May 2001 [Page 13] +Raisanen, Grotefeld expires July 2001 [Page 13] To the extent, however, that the difference between wire time and host time is uncertain, this uncertainty must be accounted for in an analysis of a given measurement method. We denote by Hsource an upper bound on the uncertainty in the difference between wire time of MP(Src) and host time on the Src host, and similarly define Hdest for the difference between the host time on the Dst host and the wire time of MP(Dst). We then note that these problems introduce a total uncertainty of Hsource+Hdest. This estimate of total wire-vs-host uncertainty should be included in the error/uncertainty analysis of any measurement implementation. @@ -658,21 +661,21 @@ remove outliers, which will be found in measuring any physical property; (2) a particular confidence level should be specified so that the results of independent implementations can be compared.} From the discussion in the previous two sections, the error in measurements could be bounded by determining all the individual uncertainties, and adding them together to form Esynch(t) + ResMP(Src) + ResMP(Dst) + Hsource + Hdest. -Raisanen, Grotefeld expires May 2001 [Page 14] +Raisanen, Grotefeld expires July 2001 [Page 14] However, reasonable bounds on both the clock-related uncertainty captured by the first three terms and the host-related uncertainty captured by the last two terms should be possible by careful design techniques and calibrating the instruments using a known, isolated, network in a lab. For example, the clock-related uncertainties are greatly reduced through the use of a GPS time source. The sum of Esynch(t) + ResMP(Src) + ResMP(Dst) is small, and is also bounded for the duration of the measurement because of the global time source. @@ -706,21 +709,21 @@ Note that random error is a function of measurement load. For example, if many paths will be measured by one instrument, this might increase interrupts, process scheduling, and disk I/O (for example, recording the measurements), all of which may increase the random error in measured singletons. Therefore, in addition to minimal load measurements to find the systematic error, calibration measurements should be performed with the same measurement load that the instruments will see in the field. -Raisanen, Grotefeld expires May 2001 [Page 15] +Raisanen, Grotefeld expires July 2001 [Page 15] We wish to reiterate that this statistical treatment refers to the calibration of the instrument; it is used to "calibrate the meter stick" and say how well the meter stick reflects reality. 4.7 Reporting the metric The calibration and context in which the metric is measured MUST be carefully considered, and SHOULD always be reported along with metric results. We now present five items to consider: the Type-P of test packets, the threshold of delay equivalent to loss, error @@ -750,21 +753,21 @@ from the measured values. + You SHOULD also report the calibration error, e, such that the true value is the reported value plus or minus e, with 95% confidence (see the last section.) + If possible, the conditions under which a test packet with finite delay is reported as lost due to resource exhaustion on the measurement instrument SHOULD be reported. -Raisanen, Grotefeld expires May 2001 [Page 16] +Raisanen, Grotefeld expires July 2001 [Page 16] 4.7.4. Path The path traversed by the packets SHOULD be reported, if possible. In general it is impractical to know the precise path a given packet takes through the network. The precise path may be known for certain Type-P packets on short or stable paths. If Type-P includes the record route (or loose-source route) option in the IP header, and the path is short enough, and all routers* on the path support record (or loose-source) route, then the path will be precisely @@ -799,21 +802,21 @@ test 1,000 two-minute VoIP calls rather than a single 2,000 minute VoIP call. When considering collection of a sample of samples, issues like the interval between samples (e.g. Poisson vs. periodic, time of day/day of week), composition of samples (e.g. equal (Tf-T0 duration, different packet sizes), and network considerations (e.g. run different samples over different intervening link-host combinations) should be taken into account. For items like the interval between samples, the pattern of use of the application being measured should be considered. -Raisanen, Grotefeld expires May 2001 [Page 17] +Raisanen, Grotefeld expires July 2001 [Page 17] 4.9 Statistics based on Type-P-One-way-Delay-Periodic-Stream 4.9.1 Statistics calculable from one sample As a metric based on a sample representative of certain applications, some general purpose statistics (e.g. median and percentile) may be less applicable than ways to characterize the range of delay values recorded during the sample metrics. @@ -849,25 +852,25 @@ 5. Security Considerations 5.1 Denial of Service Attacks This metric generates a periodic stream of packets from one host (Src) to another host (Dst) through intervening networks. This metric could be abused for denial of service attacks directed at Dst and/or the intervening network(s). -Raisanen, Grotefeld expires May 2001 [Page 18] +Raisanen, Grotefeld expires July 2001 [Page 18] Administrators of Src, Dst, and the intervening network(s) should establish bilateral or multi-lateral agreements regarding the timing, size, and frequency of collection of sample metrics. Use of this - metric in excess the terms agreed between the participants MAY BE + metric in excess the terms agreed between the participants may BE cause for immediate rejection or discard of packets or other escalation procedures defined between the affected parties. 5.2 User data confidentiality This metric generates packets for a sample metric, rather than taking samples based on user data. Thus, this metric does not threaten user data confidentiality. 5.3 Interference with the metric @@ -875,58 +878,57 @@ It may be possible to identify that a certain packet or stream of packets are part of a sample metric. With that knowledge at Dst and/or the intervening networks, it is possible to change the processing of the packets (e.g. increasing or decreasing delay) that may distort the measured performance. It may also be possible to generate additional packets that appear to be part of the sample metric. These additional packets are likely to perturb the results of the sample measurement. To discourage the kind of interference mentioned above, packet - interference checks, such as cryptographic hash, MAY be used. + interference checks, such as cryptographic hash, may be used. 6. Acknowledgements The authors wish to thank the chairs of the IPPM WG for comments that have made the present draft clearer and more focused. Howard Stanislevic and Al Morton ahave presented useful comments and questions. The authors have also built on the substantial foundations laid by the authors of the framework for IP performance [1]. -Raisanen, Grotefeld expires May 2001 [Page 19] +Raisanen, Grotefeld expires July 2001 [Page 19] 7. References [1] V.Paxson, G.Almes, J.Mahdavi, and M.Mathis: Framework for IP Performance Metrics, IETF RFC 2330, May 1998. [2] G.Almes, S.Kalidindi, and M.Zekauskas: A one-way delay metric for IPPM, IETF RFC 2679, September 1999. [3] International Telecommunications Union recommendation I.380, February 1999. [4] S. Bradner: Key words for use in RFCs to Indicate Requirement Levels, RFC 2119, March 1997. [5] ETSI TIPHON document TS-101329-5 (to be published in July). [6] G.Almes, S.Kalidindi, and M.Zekauskas: A round-trip delay metric for IPPM, IETF RFC 2681. 8. Authors' contact information Vilho Raisanen - P.O. Box 407 - Communication Systems Laboratory - Nokia Research Center + P.O. Box 300 + Nokia Networks FIN-00045 Nokia Group Finland - Phone +358 9 4376 1 - Fax. +358 9 4376 6852 + Phone +358 9 51121 + Fax. +358 9 4376 8924 Glenn Grotefeld Motorola, Inc. 1303 E. Algonquin Road 4th Floor Schaumburg, IL 60196 USA Phone +1 847 576-5992 Fax +1 847 538-7455 - EXPIRES May 2001 + EXPIRES July 2001