Network
IP Performance Measurement Working Group                                        V. Raisanen
INTERNET-DRAFT                     V.Raisanen
Internet Draft                                                    Nokia
Expiration Date:  July 2001                                 G. Grotefeld
Document: <draft-ietf-ippm-npmps-05.txt>                    G.Grotefeld
Category: Informational                                        Motorola
                                                            January 2001
                                                               A.Morton
                                                              AT&T Labs
         Network performance measurement for with periodic streams
                       <draft-ietf-ippm-npmps-04.txt>

1.
Status of this Memo
   This document is an Internet-Draft and is in full conformance with
   all provisions of Section 10 of RFC2026. RFC2026 [1].
   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups. Note that
   other groups may also distribute working documents as Internet-
   Drafts. Internet-Drafts are draft documents valid for a maximum of
   six months and may be updated, replaced, or obsoleted made obsolete by other
   documents at any time. It is inappropriate to use Internet-Drafts as
   reference material or to cite them other than as "work in progress."
   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt
   The list of Internet-Draft shadow directories Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html
   http://www.ietf.org/shadow.html.
1. Abstract
   This memo provides information describes a periodic sampling method and relevant metrics
   for assessing the performance of IP networks. First, the Internet community. This memo does not specify
   motivates periodic sampling and addresses the question of its value
   as an Internet standard alternative to Poisson sampling described in RFC 2330. The
   benefits include applicability to active and passive measurements,
   simulation of any
   kind. Distribution constant bit rate (CBR) traffic (typical of this memo is unlimited.

2. Abstract

   This document describes a multimedia
   communication, or nearly CBR, as found with voice activity
   detection), and several instances where analysis can be simplified.
   The sampling method avoids predictability by mandating random start
   times and finite length tests. Following descriptions of the
   sampling method and sample metric suitable for application-
   level IP network transport parameters, measurement for methods
   and errors are discussed. Finally, we give additional information on
   periodic streams, such as
   VoIP or streaming multimedia over IP. In measurements including security considerations.
2. Conventions used in this document,  the reader
   is assumed to be familiar with the terminology of the  Framework for
   IP Performance Metrics RFC 2330 [1]. This document is parallel
   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to
   A One-way Delay Metric for IPPM be interpreted as described in RFC 2679 2119 [2].
   Although this document
   is based on RFC 2119 was written with protocols in mind, the delay metrics, other characteristics can be measured key words
Raisanen,Grotefeld,Morton Informational  exp. May 2002               1Network performance measurement with this approach, too. For example, packet loss rate, reordering /
   out-of sequence, and successive delay variation periodic streams         Nov 2001
   are all additional
   metrics which can be built from used in this baseline set of measurements.

Raisanen, Grotefeld           expires     July  2001              [Page 1]

3. Introduction

   This document discusses concepts relevant for similar reasons.  They are used to  application-level
   performance measurements
   ensure the results of an IP network. The original driver for
   this work is Quality measurements from two different
   implementations are comparable, and to note instances when an
   implementation could perturb the network.
3. Introduction
   This memo describes a sampling method and performance metrics
   relevant to certain applications of IP networks. The original driver
   for this work was Quality of Service of interactive periodic streams
   such as multimedia conference conferencing over IP, but the idea of application-level periodic
   sampling and measurement may have a has wider scope. In the following, interactive applicability. Interactive
   multimedia traffic is used as an example below to illustrate the
   concept.

   A
   Transmitting equal size packets (or mostly same-size packets)
   through a network at regular intervals simulates a constant bit-rate
   (CBR), or nearly CBR, streaming (hereinafter
   called periodic) CBR multimedia bit stream may be simulated by
   transmitting uniformly sized stream. Hereafter, these packets (or mostly uniformly sized
   packets) at regular intervals through the network to be evaluated.
   The
   are called periodic streams. Cases of "mostly uniformly sized same-size packets" may
   be found in applications that may use smaller packets during a portion of the stream have multiple coding methods (e.g.
   digitally coded voice comfort noise during silence periods). gaps in speech).
   In the following sections, a sampling methodology and metrics are
   presented for periodic streams. The measurement results may be used
   in derivative metrics such as average and maximum delays. The memo
   seeks to formalize periodic stream measurements to achieve
   comparable results between independent implementations.
3.1 Motivation
   As noted in the IPPM framework document [1], RFC 2330 [3], a sample metric using
   regularly spaced singleton tests has some limitations when
   considered from a general measurement point of view: only part of
   the network performance spectrum is sampled. However, from the point of view of
   application-level performance, some
   applications also sample this is actually good news as
   explained below.

   IP delivery service measurements have been discussed within the
   International Telecommunications Union (ITU). A framework for IP
   service level measurements (with references to the framework for IP limited performance [1]) that is intended to spectrum and their
   performance may be suitable of critical interest.
   Periodic sampling is useful for service planning
   has been approved as I.380 [3]. The emphasis in the ITU
   recommendation following reasons:
   * It is on applicable to passive measurements, though not explicitly
   forbidding measurement, as well as active measurements. The present contribution proposes a
   method that is usable both for service planning and end-user testing
   purposes, and is based on
   measurement.
   * An active measurements.

3.1 Terminology

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to measurement can be interpreted as described in RFC 2119 [4].
   Although RFC 2119 was written with protocols in mind, the key words
   are used in this document for similar reasons.  They are used configured to
   ensure match the results
   characteristics of measurements from two different implementations
   are comparable, media flows, and simplifies the estimation of
   application performance.
   * Measurements of many network impairments (e.g., delay variation,
   consecutive loss, reordering) are sensitive to note instances when an implementation could
   perturb the network.

Raisanen, Grotefeld           expires     July sampling
   frequency.  When the impairments themselves are time-varying (and
   the variations are somewhat rare, yet important), a constant
   sampling frequency simplifies analysis.
Raisanen,Grotefeld,Morton Informational exp.May 2002                 2Network performance measurement with periodic streams         Nov 2001              [Page 2]

3.2 Considerations related to delay

   For interactive multimedia sessions, end-to-end delay
   * Frequency Domain analysis is an
   important factor. Too large a delay reduces simplified when the quality samples are
   equally spaced.
   Simulation of the CBR flows with periodic streams encourages dense
   sampling of network performance, since typical multimedia session as perceived by the participants. One approach for
   managing end-to-end delays on an Internet path involving
   heterogeneous link layer technologies is flows have
   10 to use per-domain delay
   quotas (e.g. 50 ms for a particular IP domain). 100 packets in each second.  Dense sampling permits the
   characterization of network phenomena with short duration.
4. Periodic Sampling Methodology
   The 50 ms would
   then Framework RFC [3] points out the following potential problems
   with Periodic Sampling:
   1. The performance sampled may be included synchronized with some other
   periodic behavior, or the samples may be anticipated and the results
   manipulated. Unpredictable sampling is preferred.
   2. Active measurements can cause congestion, and periodic sampling
   might drive congestion-aware senders into a calculation of synchronized state,
   producing atypical results.
   Poisson sampling produces an end-to-end delay bound. A
   practical implementation of such as scheme ought to unbiased sample for the various IP
   performance metrics, yet there are situations where alternative
   sampling methods are advantageous (as discussed under Motivation).
   We can prescribe periodic sampling methods that address issues
   like possibility the problems
   listed above. Predictability and some forms of asymmetric delays in synchronization can
   be mitigated through the use of random start times and limited
   stream duration over a route in different
   directions, test interval. The periodic sampling
   parameters produce bias, and sensitivity of an application to delay variations in judicious selection can produce a given domain. There are several alternatives as to which kind known
   bias of
   derivative delay metric one ought interest. The total traffic generated by this or any
   sampling method should be limited to use in managing end-to-end QoS.
   This question, although very interesting, is not within avoid adverse affects on non-
   test traffic (packet size, packet rate, and sample duration and
   frequency should all be considered).
   The configuration parameters of periodic sampling are:
   +  T, the scope beginning of
   this draft and a time interval where a periodic sample is not discussed further here.

   In
   desired.
   +  dT, the following, a methodology and metric are presented duration of the interval for
   measuring media stream transport QoS in an IP domain. The
   measurement results may allowed sample start times.
   +  T0, a time that MUST be used in derivative metrics such as
   average selected at random from the interval [T,
   T+dT] to start generating packets and maximum delays. A metric is presented that is taking measurements.
   +  Tf, a standard
   way time, greater than T0, for performing a measurement irrespective stopping generation of the possible QoS
   mechanism utilized in the core network. As an example, packets
   for a QoS
   mechanism without hard guarantees, measurements sample (Tf may be used relative to
   ascertain that the "best" class gets the service that has been
   promised for the traffic class in question. Moreover, an operator
   could study T0 if desired).
   +  incT, the quality nominal duration of inter-packet interval, first bit to
   first bit.
   T0 may be drawn from a cheap, low-guarantee service
   implemented using possible slack bandwidth in other classes. Such
   measurements could uniform distribution, or T0 = T + Unif(0,dT).
   Other distributions may also be made either appropriate. Start times in studying
   successive time intervals MUST use an independent value drawn from
   the feasibility distribution. In passive measurement, the arrival of a
   new service, user media
Raisanen,Grotefeld,Morton Informational exp.May 2002                 3Network performance measurement with periodic streams         Nov 2001
   flows may have sufficient randomness, or on a regular basis.

   The present draft seeks to formalize the measurements in such a way
   that interoperable results are achieved.

3.3 Protocol level issues

   The version randomized start time of
   the Internet Protocol used in the measurement affects
   (at least) packet sizes, and should during a flow may be reported.

   Fig.1 illustrates measurements on multiple protocol levels that
   are relevant needed to meet this draft. The major focus
   requirement.
   When a mix of the present draft packet sizes is on transport quality evaluation from application point desired, passive measurements usually
   possess the sequence and statistics of
   view. However, sizes in actual use, while
   active measurements would need to properly account reproduce the intended
   distribution of sizes.
5. Sample metrics for quality effects of, e.g.,
   operating system and codec on packet voice, it periodic streams
   The sample metric presented here is beneficial to be
   able similar to measure quality at the sample metric
   Type-P-One-way-Delay-Poisson-Stream presented in RFC 2679[4].
   Singletons defined in [3] and [4] are applicable here.
5.1 Metric name
   Type-P-One-way-Delay-Periodic-Stream
5.2 Metric parameters
5.2.1 Global metric parameters
   These parameters apply in all the sub-sections that follow (5.2.2,
   5.2.3, and 5.2.4).
   Parameters that each Singleton usually includes:
   +  Src, the IP level [5]. Link layer monitoring
   provides address of a way host
   +  Dst, the IP address of accounting for link layer characteristics such
   as bit error rates.

Raisanen, Grotefeld           expires     July  2001              [Page 3]
     ---------------
     | application |
     ---------------
     |  transport  | <--
     ---------------
     |   network   | <--
     ---------------
     |    link     | <--
     ---------------
     |   physical  |
     ---------------

   Fig. 1: Different possibilities for performing measurements: a
   protocol view. Above, "application" refers to all layers above
   L4 and is not host
   +  IPV, the IP version (IPv4/IPv6) used in the OSI sense.

   In general, measurement
   +  dTloss, a time interval, the results maximum waiting time for a packet
   before declaring it lost.
   +  packet size p(j), the desired number of measurements may be influenced by
   individual application requirements/responses related to bytes in the
   following issues:

   +  Lost packets: Applications may have varying tolerance to lost
      packets.  Another consideration Type-P
   packet, where j is the distribution of lost
      packets (i.e. random or bursty). size index.
   Optional parameters:
   +  Long delays: Many applications will consider packets delayed
      longer than  PktType, any additional qualifiers (transport address)
   +  Tcons, a certain value to be equivalent to lost packets
      (i.e. real time applications).
   +  Duplicate packets: Some interval for consolidating parameters collected at
   the measurement points.
   While a number of applications may be perturbed if
      duplicate packets are received.
   +  Out-of-sequence: Some will use one packet size (j = 1),
   other applications may be perturbed if use packets
      are received out of sequence. This different sizes (j > 1).
   Especially in cases of congestion, it may be in addition useful to use packets
   smaller than the
      possibility maximum or predominant size of exceeding packets in the "long" delay threshold as a result
      of being out of sequence. An out-of-sequence packet outcome
      occurs when a single IP packet received
   periodic stream.
   A topology where Src and Dst are separate from the measurement
   points is assumed.
5.2.2 Parameters collected at a DST the measurement point has a sequence number  higher than MP(Src)
   Parameters that which is
      expected, and therefore, each Singleton usually includes:
Raisanen,Grotefeld,Morton Informational exp.May 2002                 4Network performance measurement with periodic streams         Nov 2001
   +  Tstamp(Src)[i], for each packet [i], the time of the packet is OOS due to re-ordering. as
   measured at MP(Src)
   Additional parameters:
   +  Corrupt  PktID(Src) [i], for each packet header: Most applications will probably treat a
      packet with a corrupt header as equivalent to [i], a lost packet. unique identification or
   sequence number.
   +  Corrupt packet payload: Some applications (e.g. digital voice
      codecs) may accept corrupt  PktSi(Src) [i], for each packet payload.  In some cases, [i], the actual packet payload may contain application specific forward error
      correction (FEC) that can compensate for some level of
      corruption.
   +  Spurious packet: Dst size.
   Some applications may receive spurious packets (i.e. use packets
      that are not sent by the Src as part of the metric).  Many
      applications may be perturbed by spurious packets.

   Depending, e.g., on the observed protocol level, some issues listed
   above may be indistinguishable from others by the application, it
   may be important different sizes, either
   because of application requirements or in response to preserve IP
   performance experienced.
5.2.3 Parameters collected at the distinction measurement point MP(Dst)
   +  Tstamp(Dst)[i], for each packet [i], the operators of
   Src, Dst, and/or the intermediate network(s).

Raisanen, Grotefeld           expires     July  2001              [Page 4]
   Because time of the possible errors listed above, in most cases it is
   recommended to use a packet identifier as
   measured at MP(Dst)
   +  PktID(Dst) [i], for each packet generated at
   Src. Identifiers for the metric sample may be those used by the
   underlying transport layer (e.g. RTP sequence number) [i], a unique identification or
   sequence number.
   +  PktSi(Dst) [i], for each packet [i], the same
   identifiers used by an application if the application to be modeled
   by the metric uses an identifier. The possibility of identifier
   roll-over (reuse if intentional) during a metric collected over actual packet size.
   Optional parameters:
   +  dTstop, a "long" (application dependent) time should be observed.

   If the application does not use an identifier, it may still be
   useful interval, used to add identifiers to time Tf to determine when
   to stop collecting metrics for a sample
   +  PktStatus [i], for each packet [i], the packets in status of the metric sample packet
   received.  Possible status includes OK, packet header corrupt,
   packet payload corrupt, duplicate, fragment. The criteria to
   help identify possible anomalies such as out of sequence packets.
   This
   determine the status MUST be specified, if used.
5.2.4 Sample Metrics resulting from combining parameters at MP(Src) and
MP(Dst)
   Using the parameters above, a delay singleton would be most useful in calculated as
   follows:
   +  Delay [i], for each packet [i], the case where time interval
                Delay[i] = Tstamp(Dst)[i] - Tstamp(Src)[i]
   For the application
   expects following conditions, it will not be possible to receive packets in sequence, but has no capability be able to
   identify
   compute delay singletons:
   Spurious: There will be no Tstamp(Src)[i] time
   Not received: There will be no Tstamp (Dst) [i]
   Corrupt packet header: There will be no Tstamp (Dst) [i]
   Duplicate:  Only the sequence first non-corrupt copy of packets the packet
   received at Dst.

3.4 Application-level measurement

   In what follows, a  Dst should have Delay [i] computed.
   A sample metric is proposed for application-level network
   performance measurement. In effect, the metric average delay is an emulation of
   periodic multimedia stream performance. The justification for using
   realistic application metrics in the measurement:

   +  The results of the measurement are automatically relevant as follows
               AveDelay = (1/N)Sum(from i=1 to the N, Delay[i])
   assuming all packets i= 1 though N have valid singletons.
   A delay variation [5] singleton can also be computed:
Raisanen,Grotefeld,Morton Informational exp.May 2002                 5Network performance as perceived by the application in question. measurement with periodic streams         Nov 2001
   +  All  IPDV[i], for each packet [i] except the first one, delay
   variation between successive packets in the measurement contribute to accuracy of would be calculated as
                     IPDV[I] = Delay[i] - Delay [i-1]
   IPDV[i] may be negative, zero, or positive. Delay singletons for
   packets i and i-1 must be calculable or IPDV[i] is undefined.
   An example metric for the
      estimation of performance variation at timescale that IPDV sample is
      important to the multimedia application (packetization
      interval).
   +  Effects range:
                   RangeIPDV = max(IPDV[]) - min(IPDV[])
5.3 High level description of elastic traffic (TCP) on measurement packets are
      different for a sustained stream than for single packets during
      overloading situations as discussed in [3].

3.5 Measurement types

   Delay measurements can be one-way [2,3], paired one-way, or
   round-trip [6]. Accordingly, the measurements may be performed
   either with synchronized procedure to collect a sample
   Beginning on or unsynchronized Src/Dst host clocks.
   Different possibilities after time T0, Type-P packets are listed below.

   The reference measurement setup for all measurement types is
   shown in Fig. 2.

Raisanen, Grotefeld           expires     July  2001              [Page 5]
     ----------------< IP >--------------------
     |          |                  |          |
   -------   -------           --------    --------
   | generated by Src |   | MP  |           | MP   |    |
   and sent to Dst  |
   -------   |(Src)|           |(Dst) |    --------
             -------           --------

   Fig. 2: Example setup for the metric usage.

   An example of the use of the metric until time Tf is a setup reached with a source host
   (Src), a destination host (Dst), and corresponding measurement
   points (MP(Src) and MP(Dst)) as shown in Figure 2. Separate equipment
   for measurement points may be used if having Src and/or Dst conduct
   the measurement may significantly affect the delay performance to be
   measured. MP(Src)should be placed/measured close to nominal interval
   between the egress point first bit of successive packets from Src. MP(Dst) should be placed/measure close to
   the ingress point of packets for Dst. "Close" is defined incT as a
   distance sufficiently small so that application-level performance
   characteristics measured (such as delay) can at
   MP(Src).  incT may be expected nominal due to follow a number of reasons: variation
   in packet generation at Src, clock issues (see section 5.6), etc.
   MP(Src) records the corresponding performance characteristic between Src and Dst to
   an adequate accuracy. Basic principle here is that measurement
   results parameters above only for packets with
   timestamps between MP(Src) and MP(Dst) should be including T0 and Tf having the same as required Src,
   Dst, and any other qualifiers.  MP (Dst) also records for a
   measurement packets
   with time stamps between Src T0 and Dst, within the general error margin
   target of the measurement (e.g., < 1 ms; number of lost packets is
   the same). If this is not possible, (Tf + dTstop).
   Optionally at a time Tf +  Tcons (but eventually in all cases), the difference between MP-MP
   measurement
   data from MP(Src) and Src-Dst measurement MP(Dst) are consolidated to derive  the sample
   metric results.  To prevent stopping data collection too soon,
   dTcons should preferably be systematic.

   The test setup just described fulfills two important criteria:
   1) Test is made with realistic stream metrics, emulating - for example -
   a full-duplex Voice over IP (VoIP) call.
   2) Either one-way greater than or round-trip characteristics may equal to dTstop.  Conversely, to
   keep data collection reasonably efficient, dTstop should be obtained.

   It some
   reasonable time interval  (seconds/minutes/hours), even if dTloss is also possible
   infinite or extremely long.
5.4 Discussion
   This sampling methodology is intended to have intermediate measurement points between
   MP(Src) quantify the delays and MP(Dst), but that is beyond the scope
   delay variation as experienced by multimedia streams of this document.

3.5.1 One way measurement

   In an
   application. Due to the interests definitions of specifying metrics that are as generally usable
   as possible, application-level measurements based these metrics, also packet
   loss status is recorded. The nominal interval between packets
   assesses network performance variations on one-way delays a specific time scale.
   There are used in the example metrics. The implication a number of application-level
   measurement for bi-directional applications such as interactive
   multimedia conferencing is discussed below.

   Performing a single one-way measurement only yields information on
   network behavior in one direction. Moreover, the stream at the
   network transport level does not emulate accurately factors that should be taken into account when
   collecting a full-duplex
   multimedia connection.

Raisanen, Grotefeld           expires     July  2001              [Page 6]

3.5.2 Paired one way measurement

   Paired one way delay refers sample metric of Type-P-One-way-Delay-Periodic-Stream.
   +  The interval T0 to two multimedia streams: Src Tf should be specified to Dst
   and Dst cover a long enough
   time interval to Src for represent a reasonable use of the application under
   test, yet not excessively long in the same Src context(e.g. phone calls
   last longer than 100ms, but less than one week).
   +  The nominal interval between packets (incT) and Dst. By way of example, for
   some applications, the delay performance of each one way path is
   more important than packet
   size(s) (p(j)) should not define an equivalent bit rate that exceeds
   the  round trip delay. This is capacity of the case for
   delay-limited signals such as VoIP. Possible reasons for egress port of Src, the
   difference between one-way delays is different routing ingress port of streams
   from Src to Dst vs. Dst to Src.

   For example, a paired one way Dst,
Raisanen,Grotefeld,Morton Informational exp.May 2002                 6Network performance measurement may show that Src to Dst
   has an average delay with periodic streams         Nov 2001
   or the capacity of 30ms while Dst the intervening network(s), if known. There may
   be exceptional cases to Src has an average delay
   of 120ms. To a round trip delay measurement, this example would
   look like an average of 150ms delay.  Without test the knowledge response of the
   asymmetry, we might miss a problem that the application at either
   end may have with delays averaging more than 100ms.

   Moreover, paired one way to
   overload conditions in the transport networks, but these cases
   should be strictly controlled.
   +  Real delay measurement emulates values will be positive.  Therefore, it does not make
   sense to report a full-duplex
   VoIP call more accurately than negative value as a single one-way measurement only.

3.5.3 Round trip measurement

   From the point of view of periodic multimedia streams,
   round-trip measurements have two advantages: they avoid the need real delay.  However, an
   individual zero or negative delay value might be useful as part of
   host clock synchronization  and they allow for
   a simulation of
   full-duplex connections. The former aspect means that stream when trying to discover a measurement
   is easily performed, since no special equipment or NTP setup is
   needed. The latter property means that measurement streams are
   transmitted in both directions. Thus, distribution of the measurement provides
   information delay errors.
   +  Depending on quality of service as experienced by appropriate
   application.

   The downsides of round-trip measurement are the need topology, delay values may be as low as
   100 usec to 10 msec, whereby it may be important for more
   bandwidth than an one-way test Src and more complex accounting Dst to
   synchronize very closely.  GPS systems afford one way to achieve
   synchronization to within several 10s of
   packet loss. Moreover, the stream that is returning towards the
   original sender usec.  Ordinary application
   of NTP may be more bursty than the one allow synchronization to within several msec, but this
   depends on the first "leg" stability and symmetry of delay properties among the round-trip journey. The last issue, however, means in practice
   that returning stream experiences worse QoS than the other one,
   NTP agents used, and
   the performance estimates thus obtained this delay is what we are pessimistic ones. The
   possibility of asymmetric routing and queuing must be taken into
   account during analysis of the results.

   Please note that with suitable arrangements, round-trip measurements
   may be performed using paired one trying to measure.
   +  A given methodology will have to include a way measurements.

Raisanen, Grotefeld           expires     July  2001              [Page 7]

4 Sample metric for multimedia stream simulation

   The sample metric presented here is similar to determine
   whether packet was lost or whether delay is merely very large (and
   the sample packet is yet to arrive at Dst). The global metric
   Type-P-One-way-Delay-Poisson-Stream presented in [2]. "Singletons", as
   defined in [1] and [2] parameter
   dTloss defines a time interval such that delays larger than dTloss
   are not directly used in this document because
   certain key results (such interpreted as duplicate or out of sequence packets)
   cannot be identified in losses.   {Comment: For many applications, the context of
   treatment a singleton, but only large delay as part infinite/loss will be inconsequential.  A
   TCP data packet, for example, that arrives only after several
   multiples of a sample.

4.1 Metric name

   Type-P-One-way-Delay-Periodic-Stream

4.2 Metric parameters

4.2.1 Global metric parameters

   These parameters are applicable to the metrics collected in the
   following sections (4.2.2, 4.2.3, and 4.2.4).

   +  Src, usual RTT may as well have been lost.}
5.5 Additional Methodology Aspects
   As with other Type-P-* metrics, the IP address of a host
   +  Dst, detailed methodology will depend
   on the IP address Type-P (e.g., protocol number, UDP/TCP port number, size,
   precedence).
5.6 Errors and uncertainties
   The description of a host
   +  IPV, the IP version (IPv4/IPv6) used in the any specific measurement
   +  T0, a time, for starting to generate packets method should include an
   accounting and taking
         measurements for a sample
   +  Tf, a time, greater than T0, for stopping generation analysis of packets
         for a sample
   +  incT, nominal duration various sources of inter-packet interval
   +  packet size p(j), error or uncertainty.
   The Framework RFC [3] provides general guidance on this point, but
   we note here the number of bytes in each packet of Type-P of
         size j following specifics related to periodic streams and
   delay metrics:
   +  dTloss, a time interval, used  Error due to variation of incT. The reasons for determining if a packet should this can be considered lost
   uneven process scheduling, possibly due to CPU load.
   +  Tcons, a time interval [optional]

      While a number of applications will use one packet size (j = 1),
      other applications may use packets of different sizes (j > 1).
      Especially  Errors or uncertainties due to uncertainties in cases the clocks of congestion, it may be useful the
   MP(Src) and MP(Dst) measurement points.
   +  Errors or uncertainties due to have
      packets smaller than the maximum difference between 'wire time'
   and 'host time'.
5.6.1. Errors or predominant size uncertainties related to Clocks
Raisanen,Grotefeld,Morton Informational exp.May 2002                 7Network performance measurement with periodic streams         Nov 2001
   The uncertainty in a measurement of packets one-way delay is related, in
   part, to uncertainties in the periodic stream.

4.2.2 Metrics collected at clocks of MP(Src)

   +  Tstamp(Src)[i], for each packet [i], and MP(Dst). In the time of
   following, we refer to the clock used to measure when the packet as was
   measured at MP(Src)
   +  PktID [i], for each packet [i], an identification number for the as the packet sent from Src MP(Src) clock and we refer to Dst.

Raisanen, Grotefeld           expires     July  2001              [Page 8]
   +  PktSiTy [i], for each packet [i], the packet size and/or type.
      Some applications may use packets of different size, either
      because of application requirements or in response to IP
      performance experienced.

4.2.3 Metrics collected at MP (Dst)

   +  dTstop, a time interval,  clock
   used to add to time Tf to determine measure when to
         stop collecting metrics for a sample
   +  Tstamp(Dst)[i], for each packet [i], the time of the packet as
      measured was received at MP(Dst)
   +  PktID [i], for each packet [i], an identification number for the as the packet received at Dst from Src.
   +  PktSiTy [i], for each packet [i],
   MP(Dst) clock.  Alluding to the packet size and/or type.
      Some applications may use packets of different size, either
      because notions of application requirements or in response to IP
      performance experienced. synchronization,
   accuracy, resolution, and skew, we note the following:
   +  PktStatus [i], for each packet [i],  Any error in the status of synchronization between the packet
      received.  Possible status includes: OK, packet header corrupt,
      packet payload corrupt, spurious, duplicate, out-of-sequence.

4.2.4 Metrics resulting when metrics collected at MP(Src) clock and
   the MP(Dst)
      are merged

   These parameters are only available as a complete set when clock will contribute to error in the
   parameters from delay measurement.
   We say that the preceding sections (4.2.1, 4.2.2, MP(Src) clock and 4.2.3 are
   combined.

   +  Tstamp(Src)[i], for each packet [i], the time MP(Dst) clock have a
   synchronization error of Tsynch if the MP(Src) clock is Tsynch ahead
   of the packet as
      measured at MP(Src).  This entry may be blank or noted as N/A
      for spurious packets received at MP(Dst)
   +  Tstamp(Dst)[i], for each packet [i], clock.  Thus, if we know the time value of the packet as
      measured at MP(Dst).  This entry may be blank or noted as N/A Tsynch exactly,
   we could correct for packets not received at MP(Dst), received with corrupt
      packet headers, or for duplicate packets received at MP(Dst).
   +  PktID [i], for each packet [i], an identification number for the
      the packet received.  This identification number may be corrupted
      for certain packets received at MP (Dst).
   +  PktSiTy [i], for each packet [i], the packet size and/or type.
   +  PktStatus [i], for each packet [i], clock synchronization by adding Tsynch to the status
   uncorrected value of the packet
      received.  Possible status includes: OK, packet header corrupt,
      packet payload corrupt, spurious, duplicate, out-of-sequence.

Raisanen, Grotefeld           expires     July  2001              [Page 9]
   +  Delay [i], for each packet [i], the time interval Tstamp(Dst)[i] -
      Tstamp(Src)[i].  For the following conditions, it will not be
      possible Tstamp(Src) [i].
   +  The resolution of a clock adds to be able uncertainty about any time
   measured with it.  Thus, if the MP(Src) clock has a resolution of
   10 msec, then this adds 10 msec of uncertainty to compute delay:
         Spurious: There will be no Tstamp(Src)[i] any time
         Not received: There will be no Tstamp (Dst) [i]
         Corrupt packet header: There value
   measured with it.  We will be no Tstamp (Dst) [i]
         Duplicate:  Only denote the first non-corrupt copy resolution of the packet
         received at  Dst should have Delay [i] computed.
   +  SDV[i] [optional] , for each packet [i] except the first one:
         momentary delay variation between successive packets, i.e., the
         time interval Delay[i] - Delay [i-1].  SDV[i] may be negative,
         zero, or positive. Delay for both packets i source
   clock and i+1 must be
         calculable according to the definition above or SDV[i] MP(Dst) clock as ResMP(Src) and ResMP(Dst),
   respectively.
   +  The skew of a clock is
         undefined.

4.3 High level description not so much an additional issue as it is a
   realization of the procedure to collect a sample

   Beginning on or after time T0, Type-P packets are generated
   by Src and sent to Dst until time Tf fact that Tsynch is reached with itself a nominal
   interval between the first bit function of successive packets time.
   Thus, if we attempt to measure or to bound Tsynch, this needs to
   be done periodically.  Over some periods of incT as
   measured at MP(Src).  incT may time, this function can
   be nominal due approximated as a linear function plus some higher order terms;
   in these cases, one option is to use knowledge of the linear
   component to correct the clock.  Using this correction, the residual
   Tsynch is made smaller, but remains a number source of reasons:
   variation uncertainty that
   must be accounted for.  We use the function Esynch(t) to denote an
   upper bound on the uncertainty in packet generation at Src, clock issues (see section 4.6),
   etc.

   MP(Src) records synchronization.  Thus,
   |Tsynch(t)| <= Esynch(t).
   Taking these items together, we note that naive computation
   Tstamp(Dst)[i] - Tstamp(Src) [i] will be off by Tsynch(t) +/-
   (ResMP(SRc) + ResMP(Dst)).  Using the following information only for packets with
   timestamps between and including T0 and Tf: timestamp,
   packet identifier, and packet size/type notion of each packet sent from Src
   to Dst Esynch(t), we note
   that is part these clock-related problems introduce a total uncertainty of
   Esynch(t)+ Rsource + Rdest.  This estimate of total clock-related
   uncertainty should be included in the sample.

   MP (Dst) records error/uncertainty analysis of
   any measurement implementation.
5.6.2. Errors or uncertainties related to Wire-time vs Host-time
   We would like to measure the following information only for packets with time stamps between T0 and (Tf+ dTstop): timestamp, packet identifier, when a packet size/type, is measured
   and received status of each packet received from
   Src at Dst that is part of the sample.  Optionally, time-stamped at a time Tf +
   Tcons, the data from MP(Src) and when it arrives and is time-stamped
   at MP(Dst) are consolidated and we refer to derive these as "wire times."  If timestamps are
   applied by software on Src and Dst, however, then this software can
   only directly measure the results of time between when Src generates the sample metric.

   To prevent stopping data collection too soon, dTcons should be greater
   than or equal packet
   just prior to dTstop.  Conversely, sending the test packet and when Dst has started to keep data collection
   reasonably efficient, dTstop should be some reasonable time interval
   (seconds/minutes/hours), even if dTloss is infinite or extremely long.

Raisanen, Grotefeld           expires     July
Raisanen,Grotefeld,Morton Informational exp.May 2002                 8Network performance measurement with periodic streams         Nov 2001             [Page 10]

4.4 Discussion

   The sample metric thus defined is intended to probe
   process the delays and packet after having received the delay variation as experienced by multimedia streams of
   an application. Due test packet, and we
   refer to these two points as "host times".
   To the definition of extent that the metric, also packet loss
   status of packets is recorded. The delay difference between wire time and host time is assumed to
   accurately known, this knowledge can be measured at
   transport layer level. Since a range of packet sizes used to correct for wire
   time measurements and nominal
   interval between packets is used, the method probes only a specific corrected value more accurately estimates
   the desired (host time) metric, and visa-versa.
   To the extent, however, that the difference between wire time and
   host time scale is uncertain, this uncertainty must be accounted for in an
   analysis of network QoS variations.

   There are a number of factors that should be taken into account when
   collecting a sample metric of Type-P-One-way-Delay-Periodic-Stream.

   +  T0 and (Tf + dTloss) should specify a long enough time interval to
      represent a reasonable use of given measurement method.  We denote by Hsource an
   upper bound on the application under test (e.g. do
      not provide only a 100 ms uncertainty in the difference between wire time interval for a phone call)

   +  T0
   of MP(Src) and (Tf + dTloss) should specify a host time interval that is not
      excessively long compared to on the usage of Src host, and similarly define Hdest
   for the application under test
      (e.g. do not provide a one week continuous phone call)

   +  The nominal interval difference between packets (incT) and the packet size(s)
      (p(j)) should not define an equivalent bit rate that is in excess
      of host time on the capacity of Dst host and the egress port
   wire time of Src, the ingress port MP(Dst).  We then note that these problems introduce a
   total uncertainty of Dst,
      or the carrying capacity Hsource+Hdest.  This estimate of the intervening network(s). There may total wire-vs-
   host uncertainty should be exceptional cases to test included in the response error/uncertainty
   analysis of any measurement implementation.
5.6.3. Calibration
   Generally, the application to
      overload conditions in the transport networks, but these cases
      should measured values can be strictly controlled. decomposed as follows:
      measured value = true value +  Real delay values will systematic error + random error
   If the systematic error (the constant bias in measured values) can
   be positive.  Therefore, determined, it does not make
      sense to report a negative can be compensated for in the reported results.
      reported value as a real delay.  However, an
      individual zero or negative delay = measured value might be useful as part - systematic error
   therefore
      reported value = true value + random error
   The goal of
      a stream when trying calibration is to discover a distribution of determine the delay values
      of a stream.

   +  Depending on measurement topology, delay values may be systematic and random
   error generated by the instruments themselves in as low much detail as
      100 usec to 10 msec, whereby it may
   possible.  At a minimum, a bound ("e") should be important for Src and Dst to
      synchronize very closely.  GPS systems afford one way to achieve
      synchronization found such that the
   reported value is in the range (true value - e) to within several 10s of usec.  Ordinary application (true value + e)
   at least 95 percent of NTP may allow synchronization the time.  We call "e" the calibration error
   for the measurements.  It represents the degree to within several msec, but this
      depends on which the stability and symmetry of delay properties among those
      NTP agents used, and this values
   produced by the measurement instrument are repeatable; that is, how
   closely an actual delay of 30 ms is what we are trying reported as 30 ms.
   {Comment: 95 percent was chosen due to measure. A
      combination of reasons discussed in [4],
   briefly summarized as (1) some GPS-based NTP servers and a conservatively
      designed and deployed set of other NTP servers should yield good
      results, but this confidence level is yet desirable to be tested.

   +  Reordering of packets is best discussed
   able to remove outliers, which will be found in terms of measuring any
   physical property; (2) a particular confidence level should be
   specified so that the entire
      set results of measurement packets received, i.e. should independent implementations can be addressed
   compared.}
   From the discussion in
      Sec. 4.9.1.

Raisanen, Grotefeld           expires     July  2001             [Page 11]
   +  A given methodology will have to include a way to determine
      whether packet was lost or whether delay is merely very large (and the packet is yet to arrive at Dst). The global metric parameter
      dTloss defines a time interval such that delays larger than dTloss
      are interpreted as losses.
      {Comment: Note that, for many applications of these metrics, previous two sections, the
      harm error in treating a large delay as infinite might
   measurements could be zero or very
      small.  A TCP data packet, for example, that arrives only after
      several multiples of bounded by determining all the RTT may as well have been lost.}

4.5 Additional Methodology Aspects

   As individual
   uncertainties, and adding them together to form
Raisanen,Grotefeld,Morton Informational exp.May 2002                 9Network performance measurement with other Type-P-* metrics, the detailed methodology will depend periodic streams         Nov 2001
       Esynch(t) + ResMP(Src) + ResMP(Dst) + Hsource + Hdest.
   However, reasonable bounds on both the Type-P (e.g., protocol number, UDP/TCP port number, size,
   precedence).

4.6 Errors clock-related uncertainty
   captured by the first three terms and the host-related uncertainty
   captured by the last two terms should be possible by careful design
   techniques and calibrating the instruments using a known, isolated,
   network in a lab.
   For example, the clock-related uncertainties are greatly reduced
   through the use of a GPS time source.  The description sum of any specific measurement method should include an
   accounting Esynch(t) +
   ResMP(Src) + ResMP(Dst) is small, and analysis is also bounded for the
   duration of various sources the measurement because of error or uncertainty.
   The Framework document [1] provides general guidance on this point,
   but we note here the following specifics related to periodic
   streams and delay metrics: global time source.
   The host-related uncertainties, Hsource +  Error due to variation of incT. The reasons for this can Hdest, could be e.g.
      uneven process scheduling, possibly due to CPU load.
   +  Errors bounded by
   connecting two instruments back-to-back with a high-speed serial
   link or uncertainties due to uncertainties in isolated LAN segment.  In this case, repeated measurements
   are measuring the clocks of same one-way delay.
   If the
      MP(Src) test packets are small, such a network connection has a
   minimal delay that may be approximated by zero.  The measured delay
   therefore contains only systematic and MP(Dst) measurement points.
   +  Errors or uncertainties due to random error in the difference between 'wire time'
      and 'host time'.

4.6.1. Errors or uncertainties related to Clocks
   instrumentation.  The uncertainty in a measurement "average value" of one-way delay repeated measurements is related, in
   part, to uncertainties in
   the clocks of MP(Src) systematic error, and MP(Dst). In the following, we refer to variation is the clock used random error.
   One way to measure when the packet
   was measured at MP(Src) as compute the MP(Src) clock systematic error, and we refer to the
   clock used random error to measure when the packet was received at MP(Dst) as the
   MP(Dst) clock.  Alluding a
   95% confidence is to repeat the notions experiment many times - at least
   hundreds of synchronization, accuracy,
   resolution, and skew, we note tests.  The systematic error would then be the following:

   +  Any median.
   The random error in could then be found by removing the synchronization between systematic
   error from the MP(Src) clock and measured values.  The 95% confidence interval would
   be the MP(Dst) clock will contribute range from the 2.5th percentile to error in the delay
      measurement.  We say that 97.5th percentile of
   these deviations from the MP(Src) clock and the MP(Dst)
      clock have a synchronization true value.  The calibration error "e"
   could then be taken to be the largest absolute value of Tsynch if these two
   numbers, plus the MP(Src) clock clock-related uncertainty.  {Comment: as
   described, this bound is Tsynch ahead of relatively loose since the MP(Dst) clock.  Thus, if we know uncertainties
   are added, and the absolute value of Tsynch exactly, we could correct for clock
      synchronization by adding Tsynch to the uncorrected value of
      Tstamp(Dst)[i] - Tstamp(Src) [i].

Raisanen, Grotefeld           expires     July  2001             [Page 12]
   +  The accuracy of a clock largest deviation is important only in identifying used.
   As long as the time
      at which resulting value is not a given delay was measured.  Accuracy, per se, has no
      importance to the accuracy of the measurement significant fraction of delay.  When
      computing delays, we are interested only in the differences
      between clock
   measured values, not the values themselves.

   +  The resolution of it is a clock adds to uncertainty about any time
      measured with it.  Thus, if reasonable bound.  If the MP(Src) clock has a resolution of
      10 msec, then this adds 10 msec of uncertainty to any time resulting value
      measured with it.  We will denote the resolution of the source
      clock and the MP(Dst) clock as ResMP(Src) and ResMP(Dst),
      respectively.
   +  The skew of a clock is not so much an additional issue as it
   is a
      realization significant fraction of the fact measured values, then more exact
   methods will be needed to compute the calibration error.}
   Note that Tsynch random error is itself a function of time.
      Thus, measurement load.  For
   example, if we attempt to measure or to bound Tsynch, this needs to many paths will be done periodically.  Over some periods of time, measured by one instrument, this function
      can be approximated as a linear function plus some higher order
      terms; in these cases, one option is to use knowledge of the
      linear component to correct the clock.  Using this correction,
   might increase interrupts, process scheduling, and disk I/O (for
   example, recording the
      residual Tsynch is made smaller, but remains a source measurements), all of
      uncertainty that must be accounted for.  We use which may increase the function
      Esynch(t)
   random error in measured singletons.  Therefore, in addition to denote an upper bound on
   minimal load measurements to find the uncertainty in
      synchronization.  Thus, |Tsynch(t)| <= Esynch(t).

   Taking these items together, we note that naive computation
   Tstamp(Dst)[i] - Tstamp(Src) [i] will systematic error, calibration
   measurements should be off by Tsynch(t) +/-
   (ResMP(SRc) + ResMP(Dst)).  Using performed with the notion of Esynch(t), we note same measurement load that these clock-related problems introduce a total uncertainty of
   Esynch(t)+ Rsource + Rdest.  This estimate of total clock-related
   uncertainty should be included
   the instruments will see in the error/uncertainty analysis of
   any measurement implementation.

4.6.2. Errors or uncertainties related field.
   We wish to Wire-time vs Host-time

   As we have defined one-way periodic delay, we would like reiterate that this statistical treatment refers to measure the time between when a packet is measured and time-stamped at
   MP(Src) and when
   calibration of the instrument; it arrives and is time-stamped at MP(Dst) and we
   refer used to these as "wire times."  If "calibrate the timings are themselves
   performed by software on Src meter
   stick" and Dst, however, then this software can
   only directly measure say how well the time meter stick reflects reality.
Raisanen,Grotefeld,Morton Informational exp.May 2002                10Network performance measurement with periodic streams         Nov 2001
5.6.4 Errors in incT
   The nominal interval between when Src generates the packets, incT, can vary during either
   active or passive measurements. In passive measurement, packet
   just
   headers may include a timestamp applied prior to sending most of the test packet
   protocol stack, and when Dst has started to
   process the packet after having received the test packet, and we refer actual sending time may vary due to these two points as "host times".

   To
   processor scheduling. For example, H.323 systems are required to
   have packets ready for the extent that network stack within 5 ms of their ideal
   time. There may be additional variation from the difference network between wire time the
   Src and host time is
   accurately known, this knowledge can be used the MP(Src). Active measurement systems may encounter
   similar errors, but to correct a lesser extent. These errors must be
   accounted for wire time
   measurements in some types of analysis.
5.7 Reporting
   The calibration and context in which the corrected value more accurately estimates method is used MUST be
   carefully considered, and SHOULD always be reported along with
   metric results.  We next present five items to consider: the
   desired (host time) metric.

Raisanen, Grotefeld           expires     July  2001             [Page 13]
   To Type-P
   of test packets, the extent, however, that threshold of delay equivalent to loss, error
   calibration, the difference between wire time path traversed by the test packets, and
   host time background
   conditions at Src, Dst, and the intervening networks during a
   sample. This list is uncertain, this uncertainty must not exhaustive; any additional information that
   could be accounted for useful in an
   analysis interpreting applications of a given measurement method.  We denote by Hsource an
   upper bound on the uncertainty metrics should
   also be reported.
5.7.1. Type-P
   As noted in the difference between wire time Framework document [3], the value of MP(Src) and host time a metric may
   depend on the Src host, and similarly define Hdest
   for the difference between type of IP packets used to make the host time measurement, or
   "type-P".  The value of Type-P-One-way-Periodic-Delay could change
   if the protocol (UDP or TCP), port number, size, or arrangement for
   special treatment (e.g., IP precedence or RSVP) changes.  The exact
   Type-P used to make the measurements MUST be reported.
5.7.2. Threshold for delay equivalent to loss
   In addition, the threshold for delay equivalent to loss (or
   methodology to determine this threshold) MUST be reported.
5.7.3. Calibration results
   +  If the systematic error can be determined, it SHOULD be removed
   from the measured values.
   +  You SHOULD also report the calibration error, e, such that the
   true value is the reported value plus or minus e, with 95%
   confidence (see the last section.)
   +  If possible, the conditions under which a test packet with finite
   delay is reported as lost due to resource exhaustion on the Dst
   measurement instrument SHOULD be reported.
5.7.4. Path
Raisanen,Grotefeld,Morton Informational exp.May 2002                11Network performance measurement with periodic streams         Nov 2001
   The path traversed by the packets SHOULD be reported, if possible.
   In general it is impractical to know the precise path a given packet
   takes through the network.  The precise path may be known for
   certain Type-P packets on short or stable paths. If Type-P includes
   the record route (or loose-source route) option in the IP header,
   and the path is short enough, and all routers on the path support
   record (or loose-source) route, then the path will be precisely
   recorded.
   This may be impractical because the route must be short enough, many
   routers do not support (or are not configured for) record route, and
   use of this feature would often artificially worsen the performance
   observed by removing the packet from common-case processing.
   However, partial information is still valuable context. For example,
   if a host can choose between two links (and hence two separate
   routes from Src to Dst), then the initial link used is valuable
   context.  {Comment: For example, with one commercial setup, a Src on
   one NAP can reach a Dst on another NAP by either of several
   different backbone networks.}
6. Additional discussion on periodic sampling
   Fig.1 illustrates measurements on multiple protocol levels that are
   relevant to this memo. The user's focus is on transport quality
   evaluation from application point of view. However, to properly
   separate the quality contribution of the operating system and codec
   on packet voice, for example, it is beneficial to be able to measure
   quality at IP level [6]. Link layer monitoring provides a way of
   accounting for link layer characteristics such as bit error rates.
     ---------------
     | application |
     ---------------
     |  transport  | <--
     ---------------
     |   network   | <--
     ---------------
     |    link     | <--
     ---------------
     |   physical  |
     ---------------
   Fig. 1: Different possibilities for performing measurements: a
   protocol view. Above, "application" refers to all layers above L4
   and is not used in the wire
   time of MP(Dst).  We then note that these problems introduce a total
   uncertainty of Hsource+Hdest.  This estimate OSI sense.
   In general, the results of total wire-vs-host
   uncertainty should measurements may be included in influenced by
   individual application requirements/responses related to the error/uncertainty analysis of
   any
   following issues:
Raisanen,Grotefeld,Morton Informational exp.May 2002                12Network performance measurement implementation.

4.6.3. Calibration

   Generally, with periodic streams         Nov 2001
   +  Lost packets: Applications may have varying tolerance to lost
   packets.  Another consideration is the measured values can be decomposed as follows:

      measured value = true distribution of lost packets
   (i.e. random or bursty).
   +  Long delays: Many applications will consider packets delayed
   longer than a certain value to be equivalent to lost packets
   (i.e. real time applications).
   + systematic error  Duplicate packets: Some applications may be perturbed if
   duplicate packets are received.
   + random error

   If the systematic error (the constant bias in measured values) can  Reordering: Some applications may be
   determined, it can perturbed if packets arrive
   out of sequence. This may be compensated for in the reported results.

      reported value = measured value - systematic error

   therefore

      reported value = true value + random error

   The goal of calibration is addition to determine the systematic and random
   error generated by possibility of
   exceeding the instruments themselves in as much detail "long" delay threshold as
   possible.  At a minimum, result of being out of
   sequence.
   +  Corrupt packet header: Most applications will probably treat a bound ("e") should be found such that the
   reported value is in the range (true value - e)
   packet with a corrupt header as equivalent to (true value a lost packet. + e)
   at least 95 percent of the time.  We call "e"
   Corrupt packet payload: Some applications (e.g. digital voice
   codecs) may accept corrupt packet payload.  In some cases, the calibration
   packet payload may contain application specific forward error
   correction (FEC) that can compensate for some level of
   corruption.
   +  Spurious packet: Dst may receive spurious packets (i.e. packets
   that are not sent by the measurements.  It represents the degree to which Src as part of the values
   produced metric).  Many
   applications may be perturbed by spurious packets.
   Depending, e.g., on the measurement instrument are repeatable; that is, how
   closely an actual delay of 30 ms is reported as 30 ms.  {Comment: 95
   percent was chosen due to reasons discussed in [2], briefly
   summarized as (1) observed protocol level, some confidence level is desirable to be able to
   remove outliers, which will be found in measuring any physical
   property; (2) a particular confidence level should issues listed
   above may be specified so
   that indistinguishable from others by the results of independent implementations can application, it
   may be compared.}

   From important to preserve the discussion in distinction for the previous two sections, operators of
   Src, Dst, and/or the error intermediate network(s).
6.1 Measurement applications
   This sampling method provides a way to perform measurements
   irrespective of the possible QoS mechanisms utilized in the IP
   network. As an example, for a QoS mechanism without hard guarantees,
   measurements could may be bounded by determining all the individual
   uncertainties, and adding them together used to form

       Esynch(t) + ResMP(Src) + ResMP(Dst) + Hsource + Hdest.

Raisanen, Grotefeld           expires     July  2001             [Page 14]
   However, reasonable bounds on both ascertain that the clock-related uncertainty
   captured by "best" class gets the first three terms and
   service that has been promised for the host-related uncertainty
   captured by traffic class in question.
   Moreover, an operator could study the last two terms should be quality of a cheap, low-
   guarantee service implemented using possible by careful design
   techniques and calibrating slack bandwidth in
   other classes. Such measurements could be made either in studying
   the instruments using feasibility of a known, isolated,
   network in new service, or on a lab.

   For example, regular basis.
   IP delivery service measurements have been discussed within the clock-related uncertainties are greatly reduced
   through
   International Telecommunications Union (ITU). A framework for IP
   service level measurements (with references to the use framework for IP
   performance [3]) that is intended to be suitable for service
   planning has been approved as I.380 [7]. ITU-T Recommendation I.380
   covers abstract definitions of performance metrics. This memo
   describes a GPS time source.  The sum of Esynch(t) +
   ResMP(Src) + ResMP(Dst) is small, and method that is also bounded useful both for service planning and end-
   user testing purposes, in both active and passive measurements.
   Delay measurements can be one-way [3,4], paired one-way, or round-
   trip [8]. Accordingly, the
   duration of the measurement because of the global time source.

   The host-related uncertainties, Hsource + Hdest, could measurements may be bounded by
   connecting two instruments back-to-back performed either with a high-speed serial link
Raisanen,Grotefeld,Morton Informational exp.May 2002                13Network performance measurement with periodic streams         Nov 2001
   synchronized or isolated LAN segment.  In this case, repeated measurements unsynchronized Src/Dst host clocks. Different
   possibilities are
   measuring listed below.
   The reference measurement setup for all measurement types is shown
   in Fig. 2.
     ----------------< IP >--------------------
     |          |                  |          |
   -------   -------           --------    --------
   | Src |   | MP  |           | MP   |    | Dst  |
   -------   |(Src)|           |(Dst) |    --------
             -------           --------
     Fig. 2: Example measurement setup.
   An example of the same one-way delay.

   If use of the test packets are small, such method is a network connection has setup with a
   minimal delay that may be approximated by zero.  The measured delay
   therefore contains only systematic source host
   (Src), a destination host (Dst), and random error in the
   instrumentation.  The "average value" of repeated measurements is the
   systematic error, corresponding measurement
   points (MP(Src) and MP(Dst)) as shown in Figure 2. Separate
   equipment for measurement points may be used if having Src and/or
   Dst conduct the variation is the random error.

   One way to compute the systematic error, and measurement may significantly affect the random error to a
   95% confidence is delay
   performance to repeat the experiment many times - at least
   hundreds of tests.  The systematic error would then be the median.
   The random error could then be found by removing the systematic error
   from the measured values.  The 95% confidence interval would measured. MP(Src)should be the
   range from the 2.5th percentile placed/measured close
   to the 97.5th percentile egress point  of these
   deviations packets from the true value.  The calibration error "e" could then Src. MP(Dst) should be taken
   placed/measure close to be  the largest absolute value ingress point of these two numbers, plus
   the clock-related uncertainty.  {Comment: packets for Dst.
   "Close" is defined as described, this bound a distance sufficiently small so that
   application-level performance characteristics measured (such as
   delay) can be expected to follow  the corresponding performance
   characteristic between Src and Dst to an adequate accuracy. Basic
   principle here is
   relatively loose since that measurement results between MP(Src) and
   MP(Dst) should be the uncertainties are added, same as for a measurement between Src and Dst,
   within the absolute
   value general error margin target of the largest deviation measurement (e.g., < 1
   ms; number of lost packets is used.  As long as the resulting
   value same). If this is not a significant fraction of the measured values, it is a
   reasonable bound.  If possible,
   the resulting value difference between MP-MP measurement and Src-Dst measurement
   should preferably be systematic.
   The test setup just described fulfills two important criteria: 1)
   Test is made with realistic stream metrics, emulating - for example
   - a significant fraction
   of the measured values, then more exact methods will full-duplex Voice over IP (VoIP) call. 2) Either one-way or
   round-trip characteristics may be needed obtained.
   It is also possible to
   compute the calibration error.}

   Note have intermediate measurement points between
   MP(Src) and MP(Dst), but that random error is a function beyond the scope of measurement load.  For
   example, if many paths will be measured by one instrument, this might
   increase interrupts, process scheduling, and disk I/O (for example,
   recording document.
6.1.1 One way measurement
   In the measurements), all interests of which may increase the random
   error in measured singletons.  Therefore, in addition to minimal load
   measurements to find the systematic error, calibration specifying metrics that are as generally usable
   as possible, application-level measurements
   should be performed with based on one-way delays
   are used in the same example metrics. The implication of application-
   level measurement load that the
   instruments will see for bi-directional applications such as
   interactive multimedia conferencing is discussed below.
   Performing a single one-way measurement only yields information on
   network behavior in one direction. Moreover, the field.

Raisanen, Grotefeld           expires     July stream at the
Raisanen,Grotefeld,Morton Informational exp.May 2002                14Network performance measurement with periodic streams         Nov 2001             [Page 15]
   We wish to reiterate that this statistical treatment
   network transport level does not emulate accurately a full-duplex
   multimedia connection.
6.1.2 Paired one way measurement
   Paired one way delay refers to the
   calibration of the instrument; it is used two multimedia streams: Src to "calibrate the meter
   stick" and say how well the meter stick reflects reality.

4.7 Reporting the metric

   The calibration and context in which the metric is measured MUST be
   carefully considered, Dst
   and SHOULD always be reported along with metric
   results.  We now present five items Dst to consider: Src for the Type-P same Src and Dst. By way of test
   packets, example, for some
   applications, the threshold of delay equivalent to loss, error
   calibration, the performance of each one way path traversed by the test packets, and background
   conditions at Src, Dst, and is more
   important than the intervening networks during a sample. round trip delay. This list is not exhaustive; any additional information that could be
   useful in interpreting applications of the metrics should also be
   reported.

4.7.1. Type-P

   As noted in the Framework document [1], the value of the metric may
   depend on case for delay-
   limited signals such as VoIP. Possible reasons for the type difference
   between one-way delays is different routing of IP packets used streams from Src to make the measurement, or
   "type-P".  The value of Type-P-One-way-Periodic-Delay could change
   if the protocol (UDP or TCP), port number, size, or arrangement for
   special treatment (e.g., IP precedence or RSVP) changes.  The exact
   Type-P used
   Dst vs. Dst to make the measurements MUST be accurately reported.

4.7.2. Threshold for delay equivalent Src.
   For example, a paired one way measurement may show that Src to loss

   In addition, the threshold for Dst
   has an average delay equivalent to loss (or
   methodology of 30ms while Dst to determine Src has an average delay
   of 120ms. To a round trip delay measurement, this threshold) MUST be reported.

4.7.3. Calibration results

   +  If the systematic error can be determined, it SHOULD be removed
      from example would look
   like an average of 150ms delay.  Without the measured values.

   +  You SHOULD also report knowledge of the calibration error, e, such
   asymmetry, we might miss a problem that the
      true value is the reported value plus or minus e, application at either
   end may have with 95%
      confidence (see delays averaging more than 100ms.
   Moreover, paired one way delay measurement emulates a full-duplex
   VoIP call more accurately than a single one-way measurement only.
6.1.3 Round trip measurement
   From the last section.)

   +  If possible, point of view of periodic multimedia streams, round-trip
   measurements have two advantages: they avoid the conditions under which need of host clock
   synchronization and they allow for a test packet with finite
      delay simulation of full-duplex
   communication. The former aspect means that a measurement is reported as lost due to resource exhaustion on easily
   performed, since no special equipment or NTP setup is needed. The
   latter property means that measurement streams are transmitted in
   both directions. Thus, the measurement instrument SHOULD be reported.

Raisanen, Grotefeld           expires     July  2001             [Page 16]

4.7.4. Path

   The path traversed provides information on
   quality of service as experienced by two-way applications.
   The downsides of round-trip measurement are the need for more
   bandwidth than an one-way test and more complex accounting of packet
   loss. Moreover, the packets SHOULD be reported, if possible.
   In general it stream that is impractical to know the precise path a given packet
   takes through returning towards the network.  The precise path original
   sender may be known for
   certain Type-P packets more bursty than the one on short or stable paths. If Type-P includes the record route (or loose-source route) option first "leg" of the
   round-trip journey. The last issue, however, means in practice that
   returning stream may experience worse QoS than the IP header, out-going one,
   and the path is short enough, performance estimates thus obtained are pessimistic ones.
   The possibility of asymmetric routing and all routers* on the path support
   record (or loose-source) route, then queuing must be taken into
   account during analysis of the path will results.
   Note that with suitable arrangements, round-trip measurements may be precisely
   recorded.

   This
   performed using paired one way measurements.
6.2 Statistics calculable from one sample
Raisanen,Grotefeld,Morton Informational exp.May 2002                15Network performance measurement with periodic streams         Nov 2001
   Some statistics may be impractical because particularly relevant to applications
   simulated by periodic streams, such as the route must be short enough,
   many routers do not support (or are not configured for) record route, range of delay values
   recorded during the sample.
   For example, a sample metric generates 100 packets at MP(Src) with
   the following measurements at MP(Dst):
   +  80 packets received with delay [i] <= 20 ms
   +   8 packets received with delay [i] > 20 ms
   +   5 packets received with corrupt packet headers
   +   4 packets from MP(Src) with no matching packet recorded
   at MP(Dst) (effectively lost)
   +   3 packets received with corrupt packet payload and use delay [i] <=
   20 ms
   +   2 packets that duplicate one of this feature would often artificially worsen the
   performance observed by removing 80 packets received
   correctly as indicated in the packet from common-case
   processing.  However, partial information is still valuable context. first item
   For this example, packets are considered acceptable if a host can choose between two links* (and hence two
   separate routes from Src they are
   received with less than or equal to Dst), then 20ms delays and without corrupt
   packet headers or packet payload.  In this case, the initial link used percentage of
   acceptable packets is
   valuable context.  {Comment: 80/100 = 80%.
   For example, with Merit's NetNow setup,
   a Src on one NAP can reach a Dst on another NAP by either of several different backbone networks.}

4.7.5 Background conditions

   In many cases, application which will accept packets with corrupt
   packet payload and no delay bound (so long as the results of a sample may be influenced by conditions
   at Src, Dst, and/or any intervening networks.  Some things that may
   affect packet is
   received), the results percentage of a sample include:  traffic levels and/or bursts
   during the sample, link and/or host failures, etc.  Information about
   the background conditions may only be available by non-Internet means
   (e.g. phone calls, television) and may only become available days after acceptable packets is (80+8+3)/100 =
   91%.
6.3 Statistics calculable from multiple samples are taken.

4.8 Single sample vs. a "sample of samples"

   Because this metric represents a periodic stream as one sample, there
   There may be value in running multiple tests using this metric method to
   collect a "sample of samples".  For example, it may be more
   appropriate to
   test simulate 1,000 two-minute VoIP calls rather than a
   single 2,000 minute
   VoIP call.  When considering collection of a sample of multiple
   samples, issues like the interval between samples (e.g. Poisson vs. periodic, time of
   day/day of week), minutes,
   hours), composition of samples (e.g. equal (Tf-T0 Tf-T0 duration, different
   packet sizes), and network considerations (e.g. run different
   samples over different intervening link-host combinations) should be
   taken into account.  For items like the interval between samples,
   the pattern of use of the application being measured should be
   considered.

Raisanen, Grotefeld           expires     July  2001             [Page 17]

4.9 Statistics based on Type-P-One-way-Delay-Periodic-Stream

4.9.1 Statistics calculable from one sample

   As a metric based on a sample representative usage pattern for the application of certain
   applications, some interest should be
   considered.
   When computing statistics for multiple samples, more general purpose
   statistics (e.g. median and
   percentile) median, percentile, etc.) may have relevance with a
   larger number of packets.
6.4 Background conditions
   In many cases, the results may be less applicable than ways to characterize influenced by conditions at Src,
   Dst, and/or any intervening networks.  Factors that may affect the
   range of delay values recorded
   results include: traffic levels and/or bursts during the sample metrics.

   Example, a sample metric generates 100 packets as measured at MP(Src)
   with sample,
   link and/or host failures, etc.  Information about the following measurements at MP(Dst)

     +  80 packets received background
   conditions may only be available by external means (e.g. phone
Raisanen,Grotefeld,Morton Informational exp.May 2002                16Network performance measurement with periodic streams         Nov 2001
   calls, television) and may only become available days after samples
   are taken.
6.5 Considerations related to delay [i] <= 20 ms
     +   8 packets received with
   For interactive multimedia sessions, end-to-end delay [i] > 20 ms
     +   5 packets received with corrupt packet headers
     +   4 packets from MP(Src) with no matching packet recorded
           at MP(Dst) (effectively lost)
     +   3 packets received with corrupt packet payload and
           and is an
   important factor. Too large a delay [i] <= 20 ms
     +   2 packets that duplicate one reduces the quality of the 80 packets received
           correctly in
   multimedia session as perceived by the first line

   For participants. One approach
   for managing end-to-end delays on an Internet path involving
   heterogeneous link layer technologies is to use per-domain delay
   quotas (e.g. 50 ms for a particular IP domain). However, this example, packets are considered acceptable if they are
   received with less than or equal scheme
   has clear inefficiencies, and can over-constrain the problem of
   achieving some end-to-end delay objective. A more flexible
   implementation ought to 20ms address issues like possibility of
   asymmetric delays on paths, and without corrupt
   packet headers or packet payload.  In this case, the percentage sensitivity of acceptable packets is 80/100 = 80%.

   For a different an application which will accept packets with corrupt
   packet payload and no to
   delay bound (so long variations in a given domain. There are several alternatives
   as to the packet delay statistic one ought to use in managing end-to-end
   QoS. This question, although very interesting, is received), not within the percentage
   scope of acceptable packets this memo and is (80+8+3)/100 = 91%.

4.9.2 Statistics calculable from multiple samples

   For computing statistics, a "sample of samples" series of
   measurements may be performed. As not discussed in section 4.8, under
   these conditions, general purpose statistics (e.g. median, percentile,
   etc.) may be more relevant as a more statistically significant
   number of packets are used.

5. further here.
7. Security Considerations

5.1
7.1 Denial of Service Attacks
   This metric generates a periodic stream of packets from one host
   (Src) to another host (Dst) through intervening networks.  This metric
   method could be abused for denial of service attacks directed at Dst
   and/or the intervening network(s).

Raisanen, Grotefeld           expires     July  2001             [Page 18]
   Administrators of Src, Dst, and the intervening network(s) should
   establish bilateral or multi-lateral agreements regarding the
   timing, size, and frequency of collection of sample metrics.  Use of
   this
   metric method in excess of the terms agreed between the participants
   may BE be cause for immediate rejection or discard of packets or other
   escalation procedures defined between the affected parties.

5.2
7.2 User data confidentiality

   This metric
   Active use of this method generates packets for a sample metric, sample, rather
   than taking samples based on user data.  Thus, this metric data, and does not threaten user
   data confidentiality.

5.3 Passive measurement must restrict attention to
   the headers of interest. Since user payloads may be temporarily
   stored for length analysis, suitable precautions MUST be taken to
   keep this information safe and confidential.
7.3 Interference with the metric
   It may be possible to identify that a certain packet or stream of
   packets are is part of a sample metric. sample. With that knowledge at Dst and/or the
   intervening networks, it is possible to change the processing of the
   packets (e.g. increasing or decreasing delay) that may distort the
   measured performance.  It may also be possible to generate
Raisanen,Grotefeld,Morton Informational exp.May 2002                17Network performance measurement with periodic streams         Nov 2001
   additional packets that appear to be part of the sample metric.
   These additional packets are likely to perturb the results of the
   sample measurement.
   To discourage the kind of interference mentioned above, packet
   interference checks, such as cryptographic hash, may be used.

6. Acknowledgements used.
8. IANA Considerations
   Since this method and metric do not define a protocol or well-known
   values, there are no IANA considerations in this memo.
9. References
   1  Bradner, S., "The Internet Standards Process -- Revision 3", BCP
      9, RFC 2026, October 1996.
   2  Bradner, S.,  "Key words for use in RFCs to Indicate Requirement
      Levels", RFC 2119, March 1997.
   3  Paxson, V., Almes, G., Mahdavi, J., and Mathis, M., "Framework
      for IP Performance Metrics", RFC 2330, May 1998.
   4  Almes, G., Kalidindi, S., and Zekauskas, M., "A one-way delay
      metric for IPPM", RFC 2679, September 1999.
   5  Demichelis, C., and Chimento, P., "IP Packet Delay Variation
      Metric for IPPM", work in progress.
   6  ETSI TIPHON document TS-101329-5 (to be published in July).
   7  International Telecommunications Union, "Internet protocol data
      communication service _ IP packet transfer and availability
      performance parameters", Telecommunications Sector Recommendation
      I.380, February 1999.
   8  Almes, G., Kalidindi, S., and Zekauskas, M., "A round-trip delay
      metric for IPPM", IETF RFC 2681.
10. Acknowledgments
   The authors wish to thank the chairs of the IPPM WG (Matt Zekauskas
   and Merike Kaeo) for comments that have made the present draft
   clearer and more focused. Howard Stanislevic and Al Morton ahave Will Leland have
   also presented useful comments and questions. We also acknowledge
   Henk Uijterwaal's continued challenge to develop the motivation for
   this method. The authors have also  built on the substantial
   foundations foundation
   laid by the authors of the framework for IP performance [1].

Raisanen, Grotefeld           expires     July  2001             [Page 19]

7. References

   [1] V.Paxson, G.Almes, J.Mahdavi, and M.Mathis: Framework for IP
       Performance Metrics, IETF RFC 2330, May 1998.
   [2] G.Almes, S.Kalidindi, and M.Zekauskas: A one-way delay metric
       for IPPM, IETF RFC 2679, September 1999.
   [3] International Telecommunications Union recommendation I.380,
       February 1999.
   [4] S. Bradner: Key words for use in RFCs to Indicate Requirement
       Levels, RFC 2119, March 1997.
   [5] ETSI TIPHON document TS-101329-5 (to be published in July).
   [6] G.Almes, S.Kalidindi, and M.Zekauskas: A round-trip delay
       metric for IPPM, IETF RFC 2681.

8. Authors' contact information [3].
11. Author's Addresses
   Vilho Raisanen <Vilho.Raisanen@nokia.com>
Raisanen,Grotefeld,Morton Informational exp.May 2002                18Network performance measurement with periodic streams         Nov 2001
   Nokia Networks
   P.O. Box 300
   Nokia Networks
   FIN-00045 Nokia Group
   Finland
   Phone +358 9 51121 4376 1 Fax. +358 9 4376 8924 6852
   <Vilho.Raisanen@nokia.com>
   Glenn Grotefeld <g.grotefeld@motorola.com>
   Motorola, Inc.
   1303 E. Algonquin Road
   4th Floor
   Schaumburg,
   1501 W. Shure Drive, MS 2F1
   Arlington Heights, IL 60196 60004 USA
   Phone  +1 847 576-5992 435-0730 Fax    +1 847 538-7455

                           EXPIRES    July  2001 632-6800
   <g.grotefeld@motorola.com>
   Al Morton
   AT&T Labs
   Room D3 - 3C06
   200 Laurel Ave. South
   Middletown, NJ 07748 USA
   Phone  +1 732 420 1571  Fax +1 732 368 1192
   <acmorton@att.com>
Full Copyright Statement
   "Copyright (C) The Internet Society (date). All Rights Reserved.
   This document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implmentation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph
   are included on all such copies and derivative works. However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into languages other than
   English.
   The limited permissions granted above are perpetual and will not be
   revoked by the Internet Society or its successors or assigns.
   This document and the information contained herein is provided on an
   "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
   TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
   BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
   HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
   MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Raisanen,Grotefeld,Morton Informational exp.May 2002                19