draft-ietf-rmcat-eval-criteria-13.txt   draft-ietf-rmcat-eval-criteria-14.txt 
RMCAT WG V. Singh RMCAT WG V. Singh
Internet-Draft callstats.io Internet-Draft callstats.io
Intended status: Informational J. Ott Intended status: Informational J. Ott
Expires: September 10, 2020 Technical University of Munich Expires: September 20, 2020 Technical University of Munich
S. Holmer S. Holmer
Google Google
March 9, 2020 March 19, 2020
Evaluating Congestion Control for Interactive Real-time Media Evaluating Congestion Control for Interactive Real-time Media
draft-ietf-rmcat-eval-criteria-13 draft-ietf-rmcat-eval-criteria-14
Abstract Abstract
The Real-time Transport Protocol (RTP) is used to transmit media in The Real-time Transport Protocol (RTP) is used to transmit media in
telephony and video conferencing applications. This document telephony and video conferencing applications. This document
describes the guidelines to evaluate new congestion control describes the guidelines to evaluate new congestion control
algorithms for interactive point-to-point real-time media. algorithms for interactive point-to-point real-time media.
Status of This Memo Status of This Memo
skipping to change at page 1, line 36 skipping to change at page 1, line 36
Internet-Drafts are working documents of the Internet Engineering Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet- working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/. Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress." material or to cite them other than as "work in progress."
This Internet-Draft will expire on September 10, 2020. This Internet-Draft will expire on September 20, 2020.
Copyright Notice Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the Copyright (c) 2020 IETF Trust and the persons identified as the
document authors. All rights reserved. document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents Provisions Relating to IETF Documents
(https://trustee.ietf.org/license-info) in effect on the date of (https://trustee.ietf.org/license-info) in effect on the date of
publication of this document. Please review these documents publication of this document. Please review these documents
skipping to change at page 2, line 27 skipping to change at page 2, line 27
4.4. Loss generation model . . . . . . . . . . . . . . . . . . 7 4.4. Loss generation model . . . . . . . . . . . . . . . . . . 7
4.5. Jitter models . . . . . . . . . . . . . . . . . . . . . . 7 4.5. Jitter models . . . . . . . . . . . . . . . . . . . . . . 7
4.5.1. Random Bounded PDV (RBPDV) . . . . . . . . . . . . . 8 4.5.1. Random Bounded PDV (RBPDV) . . . . . . . . . . . . . 8
4.5.2. Approximately Random Subject to No-Reordering Bounded 4.5.2. Approximately Random Subject to No-Reordering Bounded
PDV (NR-RPVD) . . . . . . . . . . . . . . . . 9 PDV (NR-RPVD) . . . . . . . . . . . . . . . . 9
4.5.3. Recommended distribution . . . . . . . . . . . . . . 10 4.5.3. Recommended distribution . . . . . . . . . . . . . . 10
5. Traffic Models . . . . . . . . . . . . . . . . . . . . . . . 10 5. Traffic Models . . . . . . . . . . . . . . . . . . . . . . . 10
5.1. TCP traffic model . . . . . . . . . . . . . . . . . . . . 10 5.1. TCP traffic model . . . . . . . . . . . . . . . . . . . . 10
5.2. RTP Video model . . . . . . . . . . . . . . . . . . . . . 11 5.2. RTP Video model . . . . . . . . . . . . . . . . . . . . . 11
5.3. Background UDP . . . . . . . . . . . . . . . . . . . . . 11 5.3. Background UDP . . . . . . . . . . . . . . . . . . . . . 11
6. Security Considerations . . . . . . . . . . . . . . . . . . . 11 6. Security Considerations . . . . . . . . . . . . . . . . . . . 12
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12 7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12
8. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 12 8. Contributors . . . . . . . . . . . . . . . . . . . . . . . . 12
9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 12 9. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 12
10. References . . . . . . . . . . . . . . . . . . . . . . . . . 12 10. References . . . . . . . . . . . . . . . . . . . . . . . . . 13
10.1. Normative References . . . . . . . . . . . . . . . . . . 13 10.1. Normative References . . . . . . . . . . . . . . . . . . 13
10.2. Informative References . . . . . . . . . . . . . . . . . 13 10.2. Informative References . . . . . . . . . . . . . . . . . 14
Appendix A. Change Log . . . . . . . . . . . . . . . . . . . . . 14 Appendix A. Change Log . . . . . . . . . . . . . . . . . . . . . 15
A.1. Changes in draft-ietf-rmcat-eval-criteria-07 . . . . . . 15 A.1. Changes in draft-ietf-rmcat-eval-criteria-07 . . . . . . 15
A.2. Changes in draft-ietf-rmcat-eval-criteria-06 . . . . . . 15 A.2. Changes in draft-ietf-rmcat-eval-criteria-06 . . . . . . 15
A.3. Changes in draft-ietf-rmcat-eval-criteria-05 . . . . . . 15 A.3. Changes in draft-ietf-rmcat-eval-criteria-05 . . . . . . 15
A.4. Changes in draft-ietf-rmcat-eval-criteria-04 . . . . . . 15 A.4. Changes in draft-ietf-rmcat-eval-criteria-04 . . . . . . 15
A.5. Changes in draft-ietf-rmcat-eval-criteria-03 . . . . . . 15 A.5. Changes in draft-ietf-rmcat-eval-criteria-03 . . . . . . 15
A.6. Changes in draft-ietf-rmcat-eval-criteria-02 . . . . . . 15 A.6. Changes in draft-ietf-rmcat-eval-criteria-02 . . . . . . 15
A.7. Changes in draft-ietf-rmcat-eval-criteria-01 . . . . . . 15 A.7. Changes in draft-ietf-rmcat-eval-criteria-01 . . . . . . 16
A.8. Changes in draft-ietf-rmcat-eval-criteria-00 . . . . . . 16 A.8. Changes in draft-ietf-rmcat-eval-criteria-00 . . . . . . 16
A.9. Changes in draft-singh-rmcat-cc-eval-04 . . . . . . . . . 16 A.9. Changes in draft-singh-rmcat-cc-eval-04 . . . . . . . . . 16
A.10. Changes in draft-singh-rmcat-cc-eval-03 . . . . . . . . . 16 A.10. Changes in draft-singh-rmcat-cc-eval-03 . . . . . . . . . 16
A.11. Changes in draft-singh-rmcat-cc-eval-02 . . . . . . . . . 16 A.11. Changes in draft-singh-rmcat-cc-eval-02 . . . . . . . . . 16
A.12. Changes in draft-singh-rmcat-cc-eval-01 . . . . . . . . . 16 A.12. Changes in draft-singh-rmcat-cc-eval-01 . . . . . . . . . 17
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 17 Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 17
1. Introduction 1. Introduction
This memo describes the guidelines to help with evaluating new This memo describes the guidelines to help with evaluating new
congestion control algorithms for interactive point-to-point real congestion control algorithms for interactive point-to-point real
time media. The requirements for the congestion control algorithm time media. The requirements for the congestion control algorithm
are outlined in [I-D.ietf-rmcat-cc-requirements]). This document are outlined in [I-D.ietf-rmcat-cc-requirements]). This document
builds upon previous work at the IETF: Specifying New Congestion builds upon previous work at the IETF: Specifying New Congestion
Control Algorithms [RFC5033] and Metrics for the Evaluation of Control Algorithms [RFC5033] and Metrics for the Evaluation of
Congestion Control Algorithms [RFC5166]. Congestion Control Algorithms [RFC5166].
The guidelines proposed in the document are intended to help prevent The guidelines proposed in the document are intended to help prevent
a congestion collapse, promote fair capacity usage and optimize the a congestion collapse, promote fair capacity usage and optimize the
media flow's throughput. Furthermore, the proposed algorithms are media flow's throughput. Furthermore, the proposed congestion
expected to operate within the envelope of the circuit breakers control algorithms are expected to operate within the envelope of the
defined in RFC8083 [RFC8083]. circuit breakers defined in RFC8083 [RFC8083].
This document only provides the broad set of network parameters and This document only provides the broad set of network parameters and
and traffic models for evaluating a new congestion control algorithm. and traffic models for evaluating a new congestion control algorithm.
The minimal requirements for congestion control proposals is to The minimal requirements for congestion control proposals is to
produce or present results for the test scenarios described in produce or present results for the test scenarios described in
[I-D.ietf-rmcat-eval-test] (Basic Test Cases), which also defines the [I-D.ietf-rmcat-eval-test] (Basic Test Cases), which also defines the
specifics for the test cases. Additionally, proponents may produce specifics for the test cases. Additionally, proponents may produce
evaluation results for the wireless test scenarios evaluation results for the wireless test scenarios
[I-D.ietf-rmcat-wireless-tests]. [I-D.ietf-rmcat-wireless-tests].
skipping to change at page 5, line 40 skipping to change at page 5, line 40
variation over time, the measurements in the plot are at 1 second variation over time, the measurements in the plot are at 1 second
intervals. Additionally, from the logs it is possible to plot the intervals. Additionally, from the logs it is possible to plot the
histogram or CDF of packet delay. histogram or CDF of packet delay.
3.1. RTP Log Format 3.1. RTP Log Format
Having a common log format simplifies running analyses across and Having a common log format simplifies running analyses across and
comparing different measurements. The log file should be tab or comparing different measurements. The log file should be tab or
comma separated containing the following details: comma separated containing the following details:
Send or receive timestamp (unix) Send or receive timestamp (Unix): <int>.<int> -- sec.usec decimal
RTP payload type RTP payload type <int> -- decimal
SSRC SSRC <int> -- hexadecimal
RTP sequence no RTP sequence no <int> -- decimal
RTP timestamp RTP timestamp <int> -- decimal
marker bit marker bit 0|1 -- character
payload size Payload size <int> -- # bytes, decimal
If the congestion control implements, retransmissions or FEC, the Each line of the log file should be terminated with CRLF, CR, or LF
characters. Empty lines are disregarded.
If the congestion control implements retransmissions or FEC, the
evaluation should report both packet loss (before applying error- evaluation should report both packet loss (before applying error-
resilience) and residual packet loss (after applying error- resilience) and residual packet loss (after applying error-
resilience). resilience).
These data should suffice to compute the media-encoding independent These data should suffice to compute the media-encoding independent
metrics described above. Use of a common log will allow simplified metrics described above. Use of a common log will allow simplified
post-processing and analysis across different implementations. post-processing and analysis across different implementations.
4. List of Network Parameters 4. List of Network Parameters
skipping to change at page 14, line 23 skipping to change at page 14, line 30
Daede, T., Norkin, A., and I. Brailovskiy, "Video Codec Daede, T., Norkin, A., and I. Brailovskiy, "Video Codec
Testing and Quality Measurement", draft-ietf-netvc- Testing and Quality Measurement", draft-ietf-netvc-
testing-09 (work in progress), January 2020. testing-09 (work in progress), January 2020.
[I-D.ietf-rmcat-eval-test] [I-D.ietf-rmcat-eval-test]
Sarker, Z., Singh, V., Zhu, X., and M. Ramalho, "Test Sarker, Z., Singh, V., Zhu, X., and M. Ramalho, "Test
Cases for Evaluating RMCAT Proposals", draft-ietf-rmcat- Cases for Evaluating RMCAT Proposals", draft-ietf-rmcat-
eval-test-10 (work in progress), May 2019. eval-test-10 (work in progress), May 2019.
[I-D.ietf-rmcat-wireless-tests] [I-D.ietf-rmcat-wireless-tests]
Sarker, Z., Johansson, I., Zhu, X., Fu, J., Tan, W., and Sarker, Z., Zhu, X., and J. Fu, "Evaluation Test Cases for
M. Ramalho, "Evaluation Test Cases for Interactive Real- Interactive Real-Time Media over Wireless Networks",
Time Media over Wireless Networks", draft-ietf-rmcat- draft-ietf-rmcat-wireless-tests-11 (work in progress),
wireless-tests-09 (work in progress), February 2020. March 2020.
[RFC5033] Floyd, S. and M. Allman, "Specifying New Congestion [RFC5033] Floyd, S. and M. Allman, "Specifying New Congestion
Control Algorithms", BCP 133, RFC 5033, Control Algorithms", BCP 133, RFC 5033,
DOI 10.17487/RFC5033, August 2007, DOI 10.17487/RFC5033, August 2007,
<https://www.rfc-editor.org/info/rfc5033>. <https://www.rfc-editor.org/info/rfc5033>.
[RFC5166] Floyd, S., Ed., "Metrics for the Evaluation of Congestion [RFC5166] Floyd, S., Ed., "Metrics for the Evaluation of Congestion
Control Mechanisms", RFC 5166, DOI 10.17487/RFC5166, March Control Mechanisms", RFC 5166, DOI 10.17487/RFC5166, March
2008, <https://www.rfc-editor.org/info/rfc5166>. 2008, <https://www.rfc-editor.org/info/rfc5166>.
 End of changes. 13 change blocks. 
25 lines changed or deleted 28 lines changed or added

This html diff was produced by rfcdiff 1.47. The latest version is available from http://tools.ietf.org/tools/rfcdiff/