File: rfc3918.html

package info (click to toggle)
doc-rfc 20230121-1
links: PTS, VCS
area: non-free
in suites: bookworm, forky, sid, trixie
size: 1,609,944 kB
file content (1733 lines) | stat: -rw-r--r-- 79,190 bytes
parent folder | download | duplicates (2)
<pre>Network Working Group                                           D. Stopp
Request for Comments: 3918                                          Ixia
Category: Informational                                       B. Hickman
                                                  Spirent Communications
                                                            October 2004


               <span class="h1">Methodology for IP Multicast Benchmarking</span>

Status of this Memo

   This memo provides information for the Internet community.  It does
   not specify an Internet standard of any kind.  Distribution of this
   memo is unlimited.

Copyright Notice

   Copyright (C) The Internet Society (2004).

Abstract

   The purpose of this document is to describe methodology specific to
   the benchmarking of multicast IP forwarding devices.  It builds upon
   the tenets set forth in <a href="./rfc2544">RFC 2544</a>, <a href="./rfc2432">RFC 2432</a> and other IETF
   Benchmarking Methodology Working Group (BMWG) efforts.  This document
   seeks to extend these efforts to the multicast paradigm.

   The BMWG produces two major classes of documents: Benchmarking
   Terminology documents and Benchmarking Methodology documents.  The
   Terminology documents present the benchmarks and other related terms.
   The Methodology documents define the procedures required to collect
   the benchmarks cited in the corresponding Terminology documents.

Table of Contents

   <a href="#section-1">1</a>.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  <a href="#page-2">2</a>
   <a href="#section-2">2</a>.  Key Words to Reflect Requirements. . . . . . . . . . . . . . .  <a href="#page-3">3</a>
   <a href="#section-3">3</a>.  Test Set Up. . . . . . . . . . . . . . . . . . . . . . . . . .  <a href="#page-3">3</a>
       <a href="#section-3.1">3.1</a>.  Test Considerations. . . . . . . . . . . . . . . . . . .  <a href="#page-4">4</a>
             <a href="#section-3.1.1">3.1.1</a>. IGMP Support. . . . . . . . . . . . . . . . . . .  <a href="#page-5">5</a>
             <a href="#section-3.1.2">3.1.2</a>. Group Addresses . . . . . . . . . . . . . . . . .  <a href="#page-5">5</a>
             <a href="#section-3.1.3">3.1.3</a>. Frame Sizes . . . . . . . . . . . . . . . . . . .  <a href="#page-5">5</a>
             <a href="#section-3.1.4">3.1.4</a>. TTL . . . . . . . . . . . . . . . . . . . . . . .  <a href="#page-6">6</a>
             <a href="#section-3.1.5">3.1.5</a>. Trial Duration. . . . . . . . . . . . . . . . . .  <a href="#page-6">6</a>
   <a href="#section-4">4</a>.  Forwarding and Throughput. . . . . . . . . . . . . . . . . . .  <a href="#page-6">6</a>
       <a href="#section-4.1">4.1</a>.  Mixed Class Throughput . . . . . . . . . . . . . . . . .  <a href="#page-6">6</a>
       <a href="#section-4.2">4.2</a>.  Scaled Group Forwarding Matrix . . . . . . . . . . . . .  <a href="#page-8">8</a>
       <a href="#section-4.3">4.3</a>.  Aggregated Multicast Throughput. . . . . . . . . . . . .  <a href="#page-9">9</a>



<span class="grey">Stopp &amp; Hickman              Informational                      [Page 1]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-2" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


       <a href="#section-4.4">4.4</a>.  Encapsulation/Decapsulation (Tunneling) Throughput . . . <a href="#page-10">10</a>
             <a href="#section-4.4.1">4.4.1</a>. Encapsulation Throughput. . . . . . . . . . . . . <a href="#page-10">10</a>
             <a href="#section-4.4.2">4.4.2</a>. Decapsulation Throughput. . . . . . . . . . . . . <a href="#page-12">12</a>
             <a href="#section-4.4.3">4.4.3</a>. Re-encapsulation Throughput . . . . . . . . . . . <a href="#page-14">14</a>
   <a href="#section-5">5</a>.  Forwarding Latency . . . . . . . . . . . . . . . . . . . . . . <a href="#page-15">15</a>
       <a href="#section-5.1">5.1</a>.  Multicast Latency. . . . . . . . . . . . . . . . . . . . <a href="#page-16">16</a>
       <a href="#section-5.2">5.2</a>.  Min/Max Multicast Latency. . . . . . . . . . . . . . . . <a href="#page-18">18</a>
   <a href="#section-6">6</a>.  Overhead . . . . . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-19">19</a>
       <a href="#section-6.1">6.1</a>.  Group Join Delay . . . . . . . . . . . . . . . . . . . . <a href="#page-20">20</a>
       <a href="#section-6.2">6.2</a>.  Group Leave Delay. . . . . . . . . . . . . . . . . . . . <a href="#page-22">22</a>
   <a href="#section-7">7</a>.  Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-24">24</a>
       <a href="#section-7.1">7.1</a>.  Multicast Group Capacity . . . . . . . . . . . . . . . . <a href="#page-24">24</a>
   <a href="#section-8">8</a>.  Interaction. . . . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-25">25</a>
       <a href="#section-8.1">8.1</a>.  Forwarding Burdened Multicast Latency. . . . . . . . . . <a href="#page-25">25</a>
       <a href="#section-8.2">8.2</a>.  Forwarding Burdened Group Join Delay . . . . . . . . . . <a href="#page-27">27</a>
   <a href="#section-9">9</a>.  Security Considerations. . . . . . . . . . . . . . . . . . . . <a href="#page-28">28</a>
   <a href="#section-10">10</a>. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-28">28</a>
   <a href="#section-11">11</a>. Contributions. . . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-28">28</a>
   <a href="#section-12">12</a>. References . . . . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-28">28</a>
       <a href="#section-12.1">12.1</a>. Normative References . . . . . . . . . . . . . . . . . . <a href="#page-28">28</a>
       <a href="#section-12.2">12.2</a>. Informative References . . . . . . . . . . . . . . . . . <a href="#page-29">29</a>
   <a href="#section-13">13</a>. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . <a href="#page-30">30</a>
   <a href="#section-14">14</a>. Full Copyright Statement . . . . . . . . . . . . . . . . . . . <a href="#page-31">31</a>

<span class="h2"><a class="selflink" id="section-1" href="#section-1">1</a>.  Introduction</span>

   This document defines tests for measuring and reporting the
   throughput, forwarding, latency and Internet Group Management
   Protocol (IGMP) group membership characteristics of devices that
   support IP multicast protocols.  The results of these tests will
   provide the user with meaningful data on multicast performance.

   A previous document, "Terminology for IP Multicast Benchmarking"
   [<a href="#ref-Du98" title="&quot;Terminology for IP Multicast Benchmarking&quot;">Du98</a>], defined many of the terms that are used in this document.
   The terminology document should be consulted before attempting to
   make use of this document.

   This methodology will focus on one source to many destinations,
   although many of the tests described may be extended to use multiple
   source to multiple destination topologies.

   Subsequent documents may address IPv6 multicast and related multicast
   routing protocol performance.  Additional insight on IP and multicast
   networking can be found in [<a href="#ref-Hu95" title="&quot;Routing in the Internet&quot;">Hu95</a>], [<a href="#ref-Ka98" title="&quot;IP Multicasting: the Complete Guide to Interactive Corporate Networks&quot;">Ka98</a>] and [<a href="#ref-Mt98" title="&quot;Deploying IP Multicast in the Enterprise&quot;">Mt98</a>].







<span class="grey">Stopp &amp; Hickman              Informational                      [Page 2]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-3" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


<span class="h2"><a class="selflink" id="section-2" href="#section-2">2</a>.  Key Words to Reflect Requirements</span>

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in <a href="https://www.rfc-editor.org/bcp/bcp14">BCP 14</a>, <a href="./rfc2119">RFC 2119</a>
   [<a href="#ref-Br97" title="RFC 2119">Br97</a>].  <a href="./rfc2119">RFC 2119</a> defines the use of these key words to help make the
   intent of standards track documents as clear as possible.  While this
   document uses these keywords, this document is not a standards track
   document.

<span class="h2"><a class="selflink" id="section-3" href="#section-3">3</a>.  Test set up</span>

   The set of methodologies presented in this document are for single
   ingress, multiple egress multicast scenarios as exemplified by
   Figures 1 and 2.  Methodologies for multiple ingress and multiple
   egress multicast scenarios are beyond the scope of this document.

   Figure 1 shows a typical setup for an IP multicast test, with one
   source to multiple destinations.

                     +------------+         +--------------+
                     |            |         |  destination |
   +--------+        |     Egress(-)-------&gt;|    test      |
   | source |        |            |         |   port(E1)   |
   |  test  |------&gt;(|)Ingress    |         +--------------+
   |  port  |        |            |         +--------------+
   +--------+        |     Egress(-)-------&gt;|  destination |
                     |            |         |    test      |
                     |            |         |   port(E2)   |
                     |    DUT     |         +--------------+
                     |            |               . . .
                     |            |         +--------------+
                     |            |         |  destination |
                     |     Egress(-)-------&gt;|    test      |
                     |            |         |   port(En)   |
                     +------------+         +--------------+

                          Figure 1

   If the multicast metrics are to be taken across multiple devices
   forming a System Under Test (SUT), then test frames are offered to a
   single ingress interface on a device of the SUT, subsequently
   forwarded across the SUT topology, and finally forwarded to the test
   apparatus' frame-receiving components by the test egress interface(s)
   of devices in the SUT.  Figure 2 offers an example SUT test topology.
   If a SUT is tested, the test topology and all relevant configuration
   details MUST be disclosed with the corresponding test results.




<span class="grey">Stopp &amp; Hickman              Informational                      [Page 3]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-4" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


               *-----------------------------------------*
               |                                         |
   +--------+  |                     +----------------+  |  +--------+
   |        |  |   +------------+    |DUT B Egress E0(-)-|-&gt;|        |
   |        |  |   |DUT A       |---&gt;|                |  |  |        |
   | source |  |   |            |    |      Egress E1(-)-|-&gt;|  dest. |
   |  test  |--|-&gt;(-)Ingress, I |    +----------------+  |  |  test  |
   |  port  |  |   |            |    +----------------+  |  |  port  |
   |        |  |   |            |---&gt;|DUT C Egress E2(-)-|-&gt;|        |
   |        |  |   +------------+    |                |  |  |        |
   |        |  |                     |      Egress En(-)-|-&gt;|        |
   +--------+  |                     +----------------+  |  +--------+
               |                                         |
               *------------------SUT--------------------*

                                Figure 2

   Generally, the destination test ports first join the desired number
   of multicast groups by sending IGMP Group Report messages to the
   DUT/SUT.  To verify that all destination test ports successfully
   joined the appropriate groups, the source test port MUST transmit IP
   multicast frames destined for these groups.  After test completion,
   the destination test ports MAY send IGMP Leave Group messages to
   clear the IGMP table of the DUT/SUT.

   In addition, test equipment MUST validate the correct and proper
   forwarding actions of the devices they test in order to ensure the
   receipt of the frames that are involved in the test.

<span class="h3"><a class="selflink" id="section-3.1" href="#section-3.1">3.1</a>.  Test Considerations</span>

   The methodology assumes a uniform medium topology.  Issues regarding
   mixed transmission media, such as speed mismatch, headers
   differences, etc., are not specifically addressed.  Flow control, QoS
   and other non-essential traffic or traffic-affecting mechanisms
   affecting the variable under test MUST be disabled.  Modifications to
   the collection procedures might need to be made to accommodate the
   transmission media actually tested.  These accommodations MUST be
   presented with the test results.

   An actual flow of test traffic MAY be required to prime related
   mechanisms, (e.g., process RPF events, build device caches, etc.) to
   optimally forward subsequent traffic.  Therefore, prior to running
   any tests that require forwarding of multicast or unicast packets,
   the test apparatus MUST generate test traffic utilizing the same
   addressing characteristics to the DUT/SUT that will subsequently be





<span class="grey">Stopp &amp; Hickman              Informational                      [Page 4]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-5" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   used to measure the DUT/SUT response.  The test monitor should ensure
   the correct forwarding of traffic by the DUT/SUT.  The priming action
   need only be repeated to keep the associated information current.

   It is the intent of this memo to provide the methodology for basic
   characterizations regarding the forwarding of multicast packets by a
   device or simple system of devices.  These characterizations may be
   useful in illustrating the impact of device architectural features
   (e.g., message passing versus shared memory; handling multicast
   traffic as an exception by the general purpose processor versus the
   by a primary data path, etc.) in the forwarding of multicast traffic.

   It has been noted that the formation of the multicast distribution
   tree may be a significant component of multicast performance. While
   this component may be present in some of the measurements or
   scenarios presented in this memo, this memo does not seek to
   explicitly benchmark the formation of the multicast distribution
   tree.  The benchmarking of the multicast distribution tree formation
   is left as future, more targeted work specific to a given tree
   formation vehicle.

<span class="h4"><a class="selflink" id="section-3.1.1" href="#section-3.1.1">3.1.1</a>.  IGMP Support</span>

   All of the ingress and egress interfaces MUST support a version of
   IGMP.  The IGMP version on the ingress interface MUST be the same
   version of IGMP that is being tested on the egress interfaces.

   Each of the ingress and egress interfaces SHOULD be able to respond
   to IGMP queries during the test.

   Each of the ingress and egress interfaces SHOULD also send LEAVE
   (running IGMP version 2 or later) [<a href="#ref-Ca02" title="&quot;Internet Group Management Protocol, Version 3&quot;">Ca02</a>] [<a href="#ref-Fe97" title="&quot;Internet Group Management Protocol, Version 2&quot;">Fe97</a>] after each test.

<span class="h4"><a class="selflink" id="section-3.1.2" href="#section-3.1.2">3.1.2</a>.  Group Addresses</span>

   There is no restriction to the use of multicast addresses [<a href="#ref-De89" title="&quot;Host Extensions for IP Multicasting&quot;">De89</a>] to
   compose the test traffic other than those assignments imposed by
   IANA.  The IANA assignments for multicast addresses [<a href="#ref-IANA1">IANA1</a>] MUST be
   regarded for operational consistency.  Address selection does not
   need to be restricted to Administratively Scoped IP Multicast
   addresses [<a href="#ref-Me98" title="&quot;Administratively Scoped IP Multicast&quot;">Me98</a>].

<span class="h4"><a class="selflink" id="section-3.1.3" href="#section-3.1.3">3.1.3</a>.  Frame Sizes</span>

   Each test SHOULD be run with different multicast frame sizes.  For
   Ethernet, the recommended sizes are 64, 128, 256, 512, 1024, 1280,
   and 1518 byte frames.




<span class="grey">Stopp &amp; Hickman              Informational                      [Page 5]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-6" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   Other link layer technologies MAY be used.  The minimum and maximum
   frame lengths of the link layer technology in use SHOULD be tested.

   When testing with different frame sizes, the DUT/SUT configuration
   MUST remain the same.

<span class="h4"><a class="selflink" id="section-3.1.4" href="#section-3.1.4">3.1.4</a>.  TTL</span>

   The data plane test traffic should have a TTL value large enough to
   traverse the DUT/SUT.

   The TTL in IGMP control plane messages MUST be in compliance with the
   version of IGMP in use.

<span class="h4"><a class="selflink" id="section-3.1.5" href="#section-3.1.5">3.1.5</a>.  Trial Duration</span>

   The duration of the test portion of each trial SHOULD be at least 30
   seconds.  This parameter MUST be included as part of the results
   reporting for each methodology.

<span class="h2"><a class="selflink" id="section-4" href="#section-4">4</a>.  Forwarding and Throughput</span>

   This section contains the description of the tests that are related
   to the characterization of the frame forwarding of a DUT/SUT in a
   multicast environment.  Some metrics extend the concept of throughput
   presented in <a href="./rfc1242">RFC 1242</a>.  Forwarding Rate is cited in <a href="./rfc2285">RFC 2285</a> [<a href="#ref-Ma98" title="&quot;Benchmarking Terminology for LAN Switching Devices&quot;">Ma98</a>].

<span class="h3"><a class="selflink" id="section-4.1" href="#section-4.1">4.1</a>.  Mixed Class Throughput</span>

   Objective:

   To determine the throughput of a DUT/SUT when both unicast class
   frames and multicast class frames are offered simultaneously to a
   fixed number of interfaces as defined in <a href="./rfc2432">RFC 2432</a>.

   Procedure:

   Multicast and unicast traffic are mixed together in the same
   aggregated traffic stream in order to simulate a heterogeneous
   networking environment.

   The following events MUST occur before offering test traffic:

      o  All destination test ports configured to receive multicast
         traffic MUST join all configured multicast groups;
      o  The DUT/SUT MUST learn the appropriate unicast and
         multicast addresses; and




<span class="grey">Stopp &amp; Hickman              Informational                      [Page 6]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-7" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


      o  Group membership and unicast address learning MUST be
         verified through some externally observable method.

   The intended load [<a href="#ref-Ma98" title="&quot;Benchmarking Terminology for LAN Switching Devices&quot;">Ma98</a>] SHOULD be configured as alternating
   multicast class frames and unicast class frames to a single ingress
   interface.  The unicast class frames MUST be configured to transmit
   in an unweighted round-robin fashion to all of the destination ports.

   For example, with six multicast groups and 3 destination ports with
   one unicast addresses per port, the source test port will offer
   frames in the following order:

      m1  u1  m2  u2  m3  u3  m4  u1  m5  u2  m6  u3  m1 ...

      Where:

      m&lt;Number&gt; = Multicast Frame&lt;Group&gt;
      u&lt;Number&gt; = Unicast Frame&lt;Target Port&gt;

   Mixed class throughput measurement is defined in <a href="./rfc2432">RFC 2432</a> [<a href="#ref-Du98" title="&quot;Terminology for IP Multicast Benchmarking&quot;">Du98</a>].  A
   search algorithm MUST be utilized to determine the Mixed Class
   Throughput.  The ratio of unicast to multicast frames MUST remain the
   same when varying the intended load.

   Reporting Format:

   The following configuration parameters MUST be reflected in the test
   report:

      o  Frame size(s)
      o  Number of tested egress interfaces on the DUT/SUT
      o  Test duration
      o  IGMP version
      o  Total number of multicast groups
      o  Traffic distribution for unicast and multicast traffic
         classes
      o  The ratio of multicast to unicast class traffic

   The following results MUST be reflected in the test report:

      o  Mixed Class Throughput as defined in <a href="./rfc2432">RFC 2432</a> [<a href="#ref-Du98" title="&quot;Terminology for IP Multicast Benchmarking&quot;">Du98</a>],
         including: Throughput per unicast and multicast traffic
         classes.








<span class="grey">Stopp &amp; Hickman              Informational                      [Page 7]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-8" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   The Mixed Class Throughput results for each test SHOULD be reported
   in the form of a table with a row for each of the tested frame sizes
   per the recommendations in <a href="#section-3.1.3">section 3.1.3</a>.  Each row SHOULD specify
   the intended load, number of multicast frames offered, number of
   unicast frames offered and measured throughput per class.

<span class="h3"><a class="selflink" id="section-4.2" href="#section-4.2">4.2</a>.  Scaled Group Forwarding Matrix</span>

   Objective:

   To determine Forwarding Rate as a function of tested multicast groups
   for a fixed number of tested DUT/SUT ports.

   Procedure:

   This is an iterative procedure.  The destination test port(s) MUST
   join an initial number of multicast groups on the first iteration.
   All destination test ports configured to receive multicast traffic
   MUST join all configured multicast groups.  The recommended number of
   groups to join on the first iteration is 10 groups.  Multicast
   traffic is subsequently transmitted to all groups joined on this
   iteration and the forwarding rate is measured.

   The number of multicast groups joined by each destination test port
   is then incremented, or scaled, by an additional number of multicast
   groups.  The recommended granularity of additional groups to join per
   iteration is 10, although the tester MAY choose a finer granularity.
   Multicast traffic is subsequently transmitted to all groups joined
   during this iteration and the forwarding rate is measured.

   The total number of multicast groups joined MUST not exceed the
   multicast group capacity of the DUT/SUT.  The Group Capacity (<a href="#section-7.1">Section</a>
   <a href="#section-7.1">7.1</a>) results MUST be known prior to running this test.

   Reporting Format:

   The following configuration parameters MUST be reflected in the test
   report:

      o  Frame size(s)
      o  Number of tested egress interfaces on the DUT/SUT
      o  Test duration
      o  IGMP version

   The following results MUST be reflected in the test report:

      o  The total number of multicast groups joined for that
         iteration



<span class="grey">Stopp &amp; Hickman              Informational                      [Page 8]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-9" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


      o  Forwarding rate determined for that iteration

   The Scaled Group Forwarding results for each test SHOULD be reported
   in the form of a table with a row representing each iteration of the
   test.  Each row or iteration SHOULD specify the total number of
   groups joined for that iteration, offered load, total number of
   frames transmitted, total number of frames received and the aggregate
   forwarding rate determined for that iteration.

<span class="h3"><a class="selflink" id="section-4.3" href="#section-4.3">4.3</a>.  Aggregated Multicast Throughput</span>

   Objective:

   To determine the maximum rate at which none of the offered frames to
   be forwarded through N destination interfaces of the same multicast
   groups are dropped.

   Procedure:

   Offer multicast traffic at an initial maximum offered load to a fixed
   set of interfaces with a fixed number of groups at a fixed frame
   length for a fixed duration of time.  All destination test ports MUST
   join all specified multicast groups.

   If any frame loss is detected, the offered load is decreased and the
   sender will transmit again.  An iterative search algorithm MUST be
   utilized to determine the maximum offered frame rate with a zero
   frame loss.

   Each iteration will involve varying the offered load of the multicast
   traffic, while keeping the set of interfaces, number of multicast
   groups, frame length and test duration fixed, until the maximum rate
   at which none of the offered frames are dropped is determined.

   Parameters to be measured MUST include the maximum offered load at
   which no frame loss occurred.  Other offered loads MAY be measured
   for diagnostic purposes.

   Reporting Format:

   The following configuration parameters MUST be reflected in the test
   report:

      o  Frame size(s)
      o  Number of tested egress interfaces on the DUT/SUT
      o  Test duration
      o  IGMP version
      o  Total number of multicast groups



<span class="grey">Stopp &amp; Hickman              Informational                      [Page 9]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-10" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>



   The following results MUST be reflected in the test report:

      o  Aggregated Multicast Throughput as defined in <a href="./rfc2432">RFC 2432</a>
         [<a href="#ref-Du98" title="&quot;Terminology for IP Multicast Benchmarking&quot;">Du98</a>]

   The Aggregated Multicast Throughput results SHOULD be reported in the
   format of a table with a row for each of the tested frame sizes per
   the recommendations in <a href="#section-3.1.3">section 3.1.3</a>.  Each row or iteration SHOULD
   specify offered load, total number of offered frames and the measured
   Aggregated Multicast Throughput.

<span class="h3"><a class="selflink" id="section-4.4" href="#section-4.4">4.4</a>.  Encapsulation/Decapsulation (Tunneling) Throughput</span>

   This sub-section provides the description of tests related to the
   determination of throughput measurements when a DUT/SUT or a set of
   DUTs are acting as tunnel endpoints.

   For this specific testing scenario, encapsulation or tunneling refers
   to a packet that contains an unsupported protocol feature in a format
   that is supported by the DUT/SUT.

<span class="h4"><a class="selflink" id="section-4.4.1" href="#section-4.4.1">4.4.1</a>.  Encapsulation Throughput</span>

   Objective:

   To determine the maximum rate at which frames offered to one ingress
   interface of a DUT/SUT are encapsulated and correctly forwarded on
   one or more egress interfaces of the DUT/SUT without loss.

   Procedure:

     Source              DUT/SUT                Destination
    Test Port                                   Test Port(s)
   +---------+        +-----------+             +---------+
   |         |        |           |             |         |
   |         |        |     Egress|--(Tunnel)--&gt;|         |
   |         |        |           |             |         |
   |         |-------&gt;|Ingress    |             |         |
   |         |        |           |             |         |
   |         |        |     Egress|--(Tunnel)--&gt;|         |
   |         |        |           |             |         |
   +---------+        +-----------+             +---------+

                         Figure 3






<span class="grey">Stopp &amp; Hickman              Informational                     [Page 10]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-11" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   Figure 3 shows the setup for testing the encapsulation throughput of
   the DUT/SUT.  One or more tunnels are created between each egress
   interface of the DUT/SUT and a destination test port.  Non-
   Encapsulated multicast traffic will then be offered by the source
   test port, encapsulated by the DUT/SUT and forwarded to the
   destination test port(s).

   The DUT/SUT SHOULD be configured such that the traffic across each
   egress interface will consist of either:

      a) A single tunnel encapsulating one or more multicast address
         groups OR
      b) Multiple tunnels, each encapsulating one or more multicast
         address groups.

   The number of multicast groups per tunnel MUST be the same when the
   DUT/SUT is configured in a multiple tunnel configuration.  In
   addition, it is RECOMMENDED to test with the same number of tunnels
   on each egress interface.  All destination test ports MUST join all
   multicast group addresses offered by the source test port.  Each
   egress interface MUST be configured with the same MTU.

   Note: when offering large frames sizes, the encapsulation process may
   require the DUT/SUT to fragment the IP datagrams prior to being
   forwarded on the egress interface.  It is RECOMMENDED to limit the
   offered frame size such that no fragmentation is required by the
   DUT/SUT.

   A search algorithm MUST be utilized to determine the encapsulation
   throughput as defined in [<a href="#ref-Du98" title="&quot;Terminology for IP Multicast Benchmarking&quot;">Du98</a>].

   Reporting Format:

   The following configuration parameters MUST be reflected in the test
   report:

      o  Number of tested egress interfaces on the DUT/SUT
      o  Test duration
      o  IGMP version
      o  Total number of multicast groups
      o  MTU size of DUT/SUT interfaces
      o  Originating un-encapsulated frame size
      o  Number of tunnels per egress interface
      o  Number of multicast groups per tunnel
      o  Encapsulation algorithm or format used to tunnel the
         packets





<span class="grey">Stopp &amp; Hickman              Informational                     [Page 11]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-12" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   The following results MUST be reflected in the test report:

      o  Measured Encapsulated Throughput as defined in <a href="./rfc2432">RFC 2432</a>
         [<a href="#ref-Du98" title="&quot;Terminology for IP Multicast Benchmarking&quot;">Du98</a>]
      o  Encapsulated frame size

   The Encapsulated Throughput results SHOULD be reported in the form of
   a table and specific to this test there SHOULD be rows for each
   originating un-encapsulated frame size.  Each row or iteration SHOULD
   specify the offered load, encapsulation method, encapsulated frame
   size, total number of offered frames, and the encapsulation
   throughput.

<span class="h4"><a class="selflink" id="section-4.4.2" href="#section-4.4.2">4.4.2</a>.  Decapsulation Throughput</span>

   Objective:

   To determine the maximum rate at which frames offered to one ingress
   interface of a DUT/SUT are decapsulated and correctly forwarded by
   the DUT/SUT on one or more egress interfaces without loss.

   Procedure:

     Source                  DUT/SUT            Destination
    Test Port                                   Test Port(s)
   +---------+             +-----------+        +---------+
   |         |             |           |        |         |
   |         |             |     Egress|-------&gt;|         |
   |         |             |           |        |         |
   |         |--(Tunnel)--&gt;|Ingress    |        |         |
   |         |             |           |        |         |
   |         |             |     Egress|-------&gt;|         |
   |         |             |           |        |         |
   +---------+             +-----------+        +---------+

                             Figure 4

   Figure 4 shows the setup for testing the decapsulation throughput of
   the DUT/SUT.  One or more tunnels are created between the source test
   port and the DUT/SUT.  Encapsulated multicast traffic will then be
   offered by the source test port, decapsulated by the DUT/SUT and
   forwarded to the destination test port(s).









<span class="grey">Stopp &amp; Hickman              Informational                     [Page 12]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-13" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   The DUT/SUT SHOULD be configured such that the traffic across the
   ingress interface will consist of either:

      a) A single tunnel encapsulating one or more multicast address
         groups OR
      b) Multiple tunnels, each encapsulating one or more multicast
         address groups.

   The number of multicast groups per tunnel MUST be the same when the
   DUT/SUT is configured in a multiple tunnel configuration.  All
   destination test ports MUST join all multicast group addresses
   offered by the source test port.  Each egress interface MUST be
   configured with the same MTU.

   A search algorithm MUST be utilized to determine the decapsulation
   throughput as defined in [<a href="#ref-Du98" title="&quot;Terminology for IP Multicast Benchmarking&quot;">Du98</a>].

   When making performance comparisons between the encapsulation and
   decapsulation process of the DUT/SUT, the offered frame sizes SHOULD
   reflect the encapsulated frame sizes reported in the encapsulation
   test (See <a href="#section-4.4.1">section 4.4.1</a>) in place of those noted in <a href="#section-3.1.3">section 3.1.3</a>.

   Reporting Format:

   The following configuration parameters MUST be reflected in the test
   report:

      o  Number of tested egress interfaces on the DUT/SUT
      o  Test duration
      o  IGMP version
      o  Total number of multicast groups
      o  Originating encapsulation algorithm or format used to
         tunnel the packets
      o  Originating encapsulated frame size
      o  Number of tunnels
      o  Number of multicast groups per tunnel

   The following results MUST be reflected in the test report:

      o  Measured Decapsulated Throughput as defined in <a href="./rfc2432">RFC 2432</a>
         [<a href="#ref-Du98" title="&quot;Terminology for IP Multicast Benchmarking&quot;">Du98</a>]
      o  Decapsulated frame size

   The Decapsulated Throughput results SHOULD be reported in the format
   of a table and specific to this test there SHOULD be rows for each
   originating encapsulated frame size.  Each row or iteration SHOULD
   specify the offered load, decapsulated frame size, total number of
   offered frames and the decapsulation throughput.



<span class="grey">Stopp &amp; Hickman              Informational                     [Page 13]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-14" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


<span class="h4"><a class="selflink" id="section-4.4.3" href="#section-4.4.3">4.4.3</a>.  Re-encapsulation Throughput</span>

   Objective:

   To determine the maximum rate at which frames of one encapsulated
   format offered to one ingress interface of a DUT/SUT are converted to
   another encapsulated format and correctly forwarded by the DUT/SUT on
   one or more egress interfaces without loss.

   Procedure:

     Source                DUT/SUT             Destination
    Test Port                                  Test Port(s)
   +---------+           +---------+           +---------+
   |         |           |         |           |         |
   |         |           |   Egress|-(Tunnel)-&gt;|         |
   |         |           |         |           |         |
   |         |-(Tunnel)-&gt;|Ingress  |           |         |
   |         |           |         |           |         |
   |         |           |   Egress|-(Tunnel)-&gt;|         |
   |         |           |         |           |         |
   +---------+           +---------+           +---------+

                          Figure 5

   Figure 5 shows the setup for testing the Re-encapsulation throughput
   of the DUT/SUT.  The source test port will offer encapsulated traffic
   of one type to the DUT/SUT, which has been configured to re-
   encapsulate the offered frames using a different encapsulation
   format.  The DUT/SUT will then forward the re-encapsulated frames to
   the destination test port(s).

   The DUT/SUT SHOULD be configured such that the traffic across the
   ingress and each egress interface will consist of either:

      a) A single tunnel encapsulating one or more multicast address
         groups OR
      b) Multiple tunnels, each encapsulating one or more multicast
         address groups.

   The number of multicast groups per tunnel MUST be the same when the
   DUT/SUT is configured in a multiple tunnel configuration.  In
   addition, the DUT/SUT SHOULD be configured such that the number of
   tunnels on the ingress and each egress interface are the same.  All
   destination test ports MUST join all multicast group addresses
   offered by the source test port.  Each egress interface MUST be
   configured with the same MTU.




<span class="grey">Stopp &amp; Hickman              Informational                     [Page 14]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-15" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   Note that when offering large frames sizes, the encapsulation process
   may require the DUT/SUT to fragment the IP datagrams prior to being
   forwarded on the egress interface.  It is RECOMMENDED to limit the
   offered frame sizes, such that no fragmentation is required by the
   DUT/SUT.

   A search algorithm MUST be utilized to determine the re-encapsulation
   throughput as defined in [<a href="#ref-Du98" title="&quot;Terminology for IP Multicast Benchmarking&quot;">Du98</a>].

   Reporting Format:

   The following configuration parameters MUST be reflected in the test
   report:

      o  Number of tested egress interfaces on the DUT/SUT
      o  Test duration
      o  IGMP version
      o  Total number of multicast groups
      o  MTU size of DUT/SUT interfaces
      o  Originating encapsulation algorithm or format used to
         tunnel the packets
      o  Re-encapsulation algorithm or format used to tunnel the
         packets
      o  Originating encapsulated frame size
      o  Number of tunnels per interface
      o  Number of multicast groups per tunnel

   The following results MUST be reflected in the test report:

      o  Measured Re-encapsulated Throughput as defined in <a href="./rfc2432">RFC 2432</a>
         [<a href="#ref-Du98" title="&quot;Terminology for IP Multicast Benchmarking&quot;">Du98</a>]
      o  Re-encapsulated frame size

   The Re-encapsulated Throughput results SHOULD be reported in the
   format of a table and specific to this test there SHOULD be rows for
   each originating encapsulated frame size.  Each row or iteration
   SHOULD specify the offered load, Re-encapsulated frame size, total
   number of offered frames, and the Re-encapsulated Throughput.

<span class="h2"><a class="selflink" id="section-5" href="#section-5">5</a>.  Forwarding Latency</span>

   This section presents methodologies relating to the characterization
   of the forwarding latency of a DUT/SUT in a multicast environment.
   It extends the concept of latency characterization presented in <a href="./rfc2544">RFC</a>
   <a href="./rfc2544">2544</a>.






<span class="grey">Stopp &amp; Hickman              Informational                     [Page 15]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-16" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   The offered load accompanying the latency-measured packet can affect
   the DUT/SUT packet buffering, which may subsequently impact measured
   packet latency.  This SHOULD be a consideration when selecting the
   intended load for the described methodologies below.

   <a href="./rfc1242">RFC 1242</a> and <a href="./rfc2544">RFC 2544</a> draw a distinction between device types: "store
   and forward" and "bit-forwarding."  Each type impacts how latency is
   collected and subsequently presented.  See the related RFCs for more
   information.

<span class="h3"><a class="selflink" id="section-5.1" href="#section-5.1">5.1</a>.  Multicast Latency</span>

   Objective:

   To produce a set of multicast latency measurements from a single,
   multicast ingress interface of a DUT/SUT through multiple, egress
   multicast interfaces of that same DUT/SUT as provided for by the
   metric "Multicast Latency" in <a href="./rfc2432">RFC 2432</a> [<a href="#ref-Du98" title="&quot;Terminology for IP Multicast Benchmarking&quot;">Du98</a>].

   The procedures below draw from the collection methodology for latency
   in <a href="./rfc2544">RFC 2544</a> [<a href="#ref-Br96" title="&quot;Benchmarking Methodology for Network Interconnect Devices&quot;">Br96</a>].  The methodology addresses two topological
   scenarios: one for a single device (DUT) characterization; a second
   scenario is presented or multiple device (SUT) characterization.

   Procedure:

   If the test trial is to characterize latency across a single Device
   Under Test (DUT), an example test topology might take the form of
   Figure 1 in <a href="#section-3">section 3</a>.  That is, a single DUT with one ingress
   interface receiving the multicast test traffic from frame-
   transmitting component of the test apparatus and n egress interfaces
   on the same DUT forwarding the multicast test traffic back to the
   frame-receiving component of the test apparatus.  Note that n
   reflects the number of TESTED egress interfaces on the DUT actually
   expected to forward the test traffic (as opposed to configured but
   untested, non-forwarding interfaces, for example).

   If the multicast latencies are to be taken across multiple devices
   forming a System Under Test (SUT), an example test topology might
   take the form of Figure 2 in <a href="#section-3">section 3</a>.

   The trial duration SHOULD be 120 seconds to be consistent with <a href="./rfc2544">RFC</a>
   <a href="./rfc2544">2544</a> [<a href="#ref-Br96" title="&quot;Benchmarking Methodology for Network Interconnect Devices&quot;">Br96</a>].  The nature of the latency measurement, "store and
   forward" or "bit forwarding", MUST be associated with the related
   test trial(s) and disclosed in the results report.






<span class="grey">Stopp &amp; Hickman              Informational                     [Page 16]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-17" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   A test traffic stream is presented to the DUT.  It is RECOMMENDED to
   offer traffic at the measured aggregated multicast throughput rate
   (<a href="#section-4.3">Section 4.3</a>).  At the mid-point of the trial's duration, the test
   apparatus MUST inject a uniquely identifiable ("tagged") frame into
   the test traffic frames being presented.  This tagged frame will be
   the basis for the latency measurements.  By "uniquely identifiable",
   it is meant that the test apparatus MUST be able to discern the
   "tagged" frame from the other frames comprising the test traffic set.
   A frame generation timestamp, Timestamp A, reflecting the completion
   of the transmission of the tagged frame by the test apparatus, MUST
   be determined.

   The test apparatus will monitor frames from the DUT's tested egress
   interface(s) for the expected tagged frame(s) and MUST record the
   time of the successful detection of a tagged frame from a tested
   egress interface with a timestamp, Timestamp B.  A set of Timestamp B
   values MUST be collected for all tested egress interfaces of the
   DUT/SUT.  See <a href="./rfc1242">RFC 1242</a> [<a href="#ref-Br91" title="&quot;Benchmarking Terminology for Network Interconnection Devices&quot;">Br91</a>] for additional discussion regarding
   store and forward devices and bit forwarding devices.

   A trial MUST be considered INVALID should any of the following
   conditions occur in the collection of the trial data:

      o  Unexpected differences between Intended Load and Offered
         Load or unexpected differences between Offered Load and the
         resulting Forwarding Rate(s) on the DUT/SUT egress ports.
      o  Forwarded test frames improperly formed or frame header
         fields improperly manipulated.
      o  Failure to forward required tagged frame(s) on all expected
         egress interfaces.
      o  Reception of tagged frames by the test apparatus more than
         5 seconds after the cessation of test traffic by the source
         test port.

   The set of latency measurements, M, composed from each latency
   measurement taken from every ingress/tested egress interface pairing
   MUST be determined from a valid test trial:

      M = { (Timestamp B(E0) - Timestamp A),
            (Timestamp B(E1) - Timestamp A), ...
            (Timestamp B(En) - Timestamp A) }

   where (E0 ... En) represents the range of all tested egress
   interfaces and Timestamp B represents a tagged frame detection event
   for a given DUT/SUT tested egress interface.

   A more continuous profile MAY be built from a series of individual
   measurements.



<span class="grey">Stopp &amp; Hickman              Informational                     [Page 17]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-18" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   Reporting Format:

   The following configuration parameters MUST be reflected in the test
   report:

      o  Frame size(s)
      o  Number of tested egress interfaces on the DUT/SUT
      o  Test duration
      o  IGMP version
      o  Offered load
      o  Total number of multicast groups

   The following results MUST be reflected in the test report:

      o  The set of all latencies with respective time units related
         to the tested ingress and each tested egress DUT/SUT
         interface.

   The time units of the presented latency MUST be uniform and with
   sufficient precision for the medium or media being tested.

   The results MAY be offered in a tabular format and should preserve
   the relationship of latency to ingress/egress interface for each
   multicast group to assist in trending across multiple trials.

<span class="h3"><a class="selflink" id="section-5.2" href="#section-5.2">5.2</a>.  Min/Max Multicast Latency</span>

   Objective:

   To determine the difference between the maximum latency measurement
   and the minimum latency measurement from a collected set of latencies
   produced by the Multicast Latency benchmark.

   Procedure:

   Collect a set of multicast latency measurements over a single test
   duration, as prescribed in <a href="#section-5.1">section 5.1</a>.  This will produce a set of
   multicast latencies, M, where M is composed of individual forwarding
   latencies between DUT frame ingress and DUT frame egress port pairs.
   E.g.:

      M = {L(I,E1),L(I,E2), ..., L(I,En)}

   where L is the latency between a tested ingress interface, I, of the
   DUT, and Ex a specific, tested multicast egress interface of the DUT.
   E1 through En are unique egress interfaces on the DUT.





<span class="grey">Stopp &amp; Hickman              Informational                     [Page 18]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-19" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   From the collected multicast latency measurements in set M, identify
   MAX(M), where MAX is a function that yields the largest latency value
   from set M.

   Identify MIN(M), when MIN is a function that yields the smallest
   latency value from set M.

   The Max/Min value is determined from the following formula:

      Result = MAX(M) - MIN(M)

   Reporting Format:

   The following configuration parameters MUST be reflected in the test
   report:

      o  Frame size(s)
      o  Number of tested egress interfaces on the DUT/SUT
      o  Test duration
      o  IGMP version
      o  Offered load
      o  Total number of multicast groups

   The following results MUST be reflected in the test report:

      o  The Max/Min value

   The following results SHOULD be reflected in the test report:

      o  The set of all latencies with respective time units related
         to the tested ingress and each tested egress DUT/SUT
         interface.

   The time units of the presented latency MUST be uniform and with
   sufficient precision for the medium or media being tested.

   The results MAY be offered in a tabular format and should preserve
   the relationship of latency to ingress/egress interface for each
   multicast group.

<span class="h2"><a class="selflink" id="section-6" href="#section-6">6</a>.  Overhead</span>

   This section presents methodology relating to the characterization of
   the overhead delays associated with explicit operations found in
   multicast environments.






<span class="grey">Stopp &amp; Hickman              Informational                     [Page 19]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-20" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


<span class="h3"><a class="selflink" id="section-6.1" href="#section-6.1">6.1</a>.  Group Join Delay</span>

   Objective:

   To determine the time duration it takes a DUT/SUT to start forwarding
   multicast frames from the time a successful IGMP group membership
   report has been issued to the DUT/SUT.

   Procedure:

   The Multicast Group Join Delay measurement may be influenced by the
   state of the Multicast Forwarding Database &lt;MFDB&gt; of the DUT/SUT. The
   states of the MFDB may be described as follows:

      o  State 0, where the MFDB does not contain the specified
         multicast group address.  In this state, the delay measurement
         includes the time the DUT/SUT requires to add the address to
         the MFDB and begin forwarding.   Delay measured from State 0
         provides information about how the DUT/SUT is able to add new
         addresses into MFDB.

      o  State 1, where the MFDB does contain the specified multicast
         group address.  In this state, the delay measurement includes
         the time the DUT/SUT requires to update the MFDB with the
         newly joined node&lt;s&gt; and begin forwarding to the new node&lt;s&gt;
         plus packet replication time.  Delay measured from State 1
         provides information about how well the DUT/SUT is able to
         update the MFDB for new nodes while transmitting packets to
         other nodes for the same IP multicast address.  Examples
         include adding a new user to an event that is being promoted
         via multicast packets.

   The methodology for the Multicast Group Join Delay measurement
   provides two alternate methods, based on the state of the MFDB, to
   measure the delay metric.  The methods MAY be used independently or
   in conjunction to provide meaningful insight into the DUT/SUT ability
   to manage the MFDB.

   Users MAY elect to use either method to determine the Multicast Group
   Join Delay; however the collection method MUST be specified as part
   of the reporting format.

   In order to minimize the variation in delay calculations as well as
   minimize burden on the DUT/SUT, the test SHOULD be performed with one
   multicast group.  In addition, all destination test ports MUST join
   the specified multicast group offered to the ingress interface of the
   DUT/SUT.




<span class="grey">Stopp &amp; Hickman              Informational                     [Page 20]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-21" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   Method A:

   Method A assumes that the Multicast Forwarding Database &lt;MFDB&gt; of the
   DUT/SUT does not contain or has not learned the specified multicast
   group address; specifically, the MFDB MUST be in State 0. In this
   scenario, the metric represents the time the DUT/SUT takes to add the
   multicast address to the MFDB and begin forwarding the multicast
   packet.  Only one ingress and one egress MUST be used to determine
   this metric.

   Prior to sending any IGMP Group Membership Reports used to calculate
   the Multicast Group Join Delay, it MUST be verified through
   externally observable means that the destination test port is not
   currently a member of the specified multicast group.  In addition, it
   MUST be verified through externally observable means that the MFDB of
   the DUT/SUT does not contain the specified multicast address.

   Method B:

   Method B assumes that the MFDB of the DUT/SUT does contain the
   specified multicast group address; specifically, the MFDB MUST be in
   State 1.  In this scenario, the metric represents the time the
   DUT/SUT takes to update the MFDB with the additional nodes and their
   corresponding interfaces and to begin forwarding the multicast
   packet.  One or more egress ports MAY be used to determine this
   metric.

   Prior to sending any IGMP Group Membership Reports used to calculate
   the Group Join Delay, it MUST be verified through externally
   observable means that the MFDB contains the specified multicast group
   address.  A single un-instrumented test port MUST be used to join the
   specified multicast group address prior to sending any test traffic.
   This port will be used only for insuring that the MFDB has been
   populated with the specified multicast group address and can
   successfully forward traffic to the un-instrumented port.

   Join Delay Calculation

   Once verification is complete, multicast traffic for the specified
   multicast group address MUST be offered to the ingress interface
   prior to the DUT/SUT receiving any IGMP Group Membership Report
   messages.  It is RECOMMENDED to offer traffic at the measured
   aggregated multicast throughput rate (<a href="#section-4.3">Section 4.3</a>).

   After the multicast traffic has been started, the destination test
   port (See Figure 1) MUST send one IGMP Group Membership Report for
   the specified multicast group.




<span class="grey">Stopp &amp; Hickman              Informational                     [Page 21]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-22" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   The join delay is the difference in time from when the IGMP Group
   Membership message is sent (timestamp A) and the first frame of the
   multicast group is forwarded to a receiving egress interface
   (timestamp B).

      Group Join delay time = timestamp B - timestamp A

   Timestamp A MUST be the time the last bit of the IGMP group
   membership report is sent from the destination test port; timestamp B
   MUST be the time the first bit of the first valid multicast frame is
   forwarded on the egress interface of the DUT/SUT.

   Reporting Format:

   The following configuration parameters MUST be reflected in the test
   report:

      o  Frame size(s)
      o  Number of tested egress interfaces on the DUT/SUT
      o  IGMP version
      o  Total number of multicast groups
      o  Offered load to ingress interface
      o  Method used to measure the join delay metric

   The following results MUST be reflected in the test report:

      o  The group join delay time in microseconds per egress
         interface(s)

   The Group Join Delay results for each test MAY be reported in the
   form of a table, with a row for each of the tested frame sizes per
   the recommendations in <a href="#section-3.1.3">section 3.1.3</a>.  Each row or iteration MAY
   specify the group join delay time per egress interface for that
   iteration.

<span class="h3"><a class="selflink" id="section-6.2" href="#section-6.2">6.2</a>.  Group Leave Delay</span>

   Objective:

   To determine the time duration it takes a DUT/SUT to cease forwarding
   multicast frames after a corresponding IGMP Leave Group message has
   been successfully offered to the DUT/SUT.









<span class="grey">Stopp &amp; Hickman              Informational                     [Page 22]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-23" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   Procedure:

   In order to minimize the variation in delay calculations as well as
   minimize burden on the DUT/SUT, the test SHOULD be performed with one
   multicast group.  In addition, all destination test ports MUST join
   the specified multicast group offered to the ingress interface of the
   DUT/SUT.

   Prior to sending any IGMP Leave Group messages used to calculate the
   group leave delay, it MUST be verified through externally observable
   means that the destination test ports are currently members of the
   specified multicast group.  If any of the egress interfaces do not
   forward validation multicast frames then the test is invalid.

   Once verification is complete, multicast traffic for the specified
   multicast group address MUST be offered to the ingress interface
   prior to receipt or processing of any IGMP Leave Group messages. It
   is RECOMMENDED to offer traffic at the measured aggregated multicast
   throughput rate (<a href="#section-4.3">Section 4.3</a>).

   After the multicast traffic has been started, each destination test
   port (See Figure 1) MUST send one IGMP Leave Group message for the
   specified multicast group.

   The leave delay is the difference in time from when the IGMP Leave
   Group message is sent (timestamp A) and the last frame of the
   multicast group is forwarded to a receiving egress interface
   (timestamp B).

           Group Leave delay time = timestamp B - timestamp A

   Timestamp A MUST be the time the last bit of the IGMP Leave Group
   message is sent from the destination test port; timestamp B MUST be
   the time the last bit of the last valid multicast frame is forwarded
   on the egress interface of the DUT/SUT.

   Reporting Format:

   The following configuration parameters MUST be reflected in the test
   report:

      o  Frame size(s)
      o  Number of tested egress interfaces on the DUT/SUT
      o  IGMP version
      o  Total number of multicast groups
      o  Offered load to ingress interface





<span class="grey">Stopp &amp; Hickman              Informational                     [Page 23]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-24" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   The following results MUST be reflected in the test report:

      o  The group leave delay time in microseconds per egress
         interface(s)

   The Group Leave Delay results for each test MAY be reported in the
   form of a table, with a row for each of the tested frame sizes per
   the recommendations in <a href="#section-3.1.3">section 3.1.3</a>.  Each row or iteration MAY
   specify the group leave delay time per egress interface for that
   iteration.

<span class="h2"><a class="selflink" id="section-7" href="#section-7">7</a>.  Capacity</span>

   This section offers a procedure relating to the identification of
   multicast group limits of a DUT/SUT.

<span class="h3"><a class="selflink" id="section-7.1" href="#section-7.1">7.1</a>.  Multicast Group Capacity</span>

   Objective:

   To determine the maximum number of multicast groups a DUT/SUT can
   support while maintaining the ability to forward multicast frames to
   all multicast groups registered to that DUT/SUT.

   Procedure:

   One or more destination test ports of DUT/SUT will join an initial
   number of multicast groups.

   After a minimum delay as measured by <a href="#section-6.1">section 6.1</a>, the source test
   ports MUST transmit to each group at a specified offered load.

   If at least one frame for each multicast group is forwarded properly
   by the DUT/SUT on each participating egress interface, the iteration
   is said to pass at the current capacity.

   For each successful iteration, each destination test port will join
   an additional user-defined number of multicast groups and the test
   repeats.  The test stops iterating when one or more of the egress
   interfaces fails to forward traffic on one or more of the configured
   multicast groups.

   Once the iteration fails, the last successful iteration is the stated
   Maximum Group Capacity result.







<span class="grey">Stopp &amp; Hickman              Informational                     [Page 24]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-25" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   Reporting Format:

   The following configuration parameters MUST be reflected in the test
   report:

      o  Frame size(s)
      o  Number of tested egress interfaces on the DUT/SUT
      o  IGMP version
      o  Offered load

   The following results MUST be reflected in the test report:

      o  The total number of multicast group addresses that were
         successfully forwarded through the DUT/SUT

   The Multicast Group Capacity results for each test SHOULD be reported
   in the form of a table, with a row for each of the tested frame sizes
   per the recommendations in <a href="#section-3.1.3">section 3.1.3</a>.  Each row or iteration
   SHOULD specify the number of multicast groups joined per destination
   interface, number of frames transmitted and number of frames received
   for that iteration.

<span class="h2"><a class="selflink" id="section-8" href="#section-8">8</a>.  Interaction</span>

   Network forwarding devices are generally required to provide more
   functionality than just the forwarding of traffic.  Moreover,
   network-forwarding devices may be asked to provide those functions in
   a variety of environments.  This section offers procedures to assist
   in the characterization of DUT/SUT behavior in consideration of
   potentially interacting factors.

<span class="h3"><a class="selflink" id="section-8.1" href="#section-8.1">8.1</a>.  Forwarding Burdened Multicast Latency</span>

   Objective:

   To produce a set of multicast latency measurements from a single
   multicast ingress interface of a DUT/SUT through multiple egress
   multicast interfaces of that same DUT/SUT as provided for by the
   metric "Multicast Latency" in <a href="./rfc2432">RFC 2432</a> [<a href="#ref-Du98" title="&quot;Terminology for IP Multicast Benchmarking&quot;">Du98</a>] while forwarding meshed
   unicast traffic.

   Procedure:

   The Multicast Latency metrics can be influenced by forcing the
   DUT/SUT to perform extra processing of packets while multicast class
   traffic is being forwarded for latency measurements.





<span class="grey">Stopp &amp; Hickman              Informational                     [Page 25]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-26" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   The Burdened Forwarding Multicast Latency test MUST follow the
   described setup for the Multicast Latency test in <a href="#section-5.1">Section 5.1</a>.  In
   addition, another set of test ports MUST be used to burden the
   DUT/SUT (burdening ports).  The burdening ports will be used to
   transmit unicast class traffic to the DUT/SUT in a fully meshed
   traffic distribution as described in <a href="./rfc2285">RFC 2285</a> [<a href="#ref-Ma98" title="&quot;Benchmarking Terminology for LAN Switching Devices&quot;">Ma98</a>].  The DUT/SUT
   MUST learn the appropriate unicast addresses and verified through
   some externally observable method.

   Perform a baseline measurement of Multicast Latency as described in
   <a href="#section-5.1">Section 5.1</a>.  After the baseline measurement is obtained, start
   transmitting the unicast class traffic at a user-specified offered
   load on the set of burdening ports and rerun the Multicast Latency
   test.  The offered load to the ingress port MUST be the same as was
   used in the baseline measurement.

   Reporting Format:

   Similar to <a href="#section-5.1">Section 5.1</a>, the following configuration parameters MUST
   be reflected in the test report:

      o  Frame size(s)
      o  Number of tested egress interfaces on the DUT/SUT
      o  Test duration
      o  IGMP version
      o  Offered load to ingress interface
      o  Total number of multicast groups
      o  Offered load to burdening ports
      o  Total number of burdening ports

   The following results MUST be reflected in the test report:

      o  The set of all latencies related to the tested ingress and
         each tested egress DUT/SUT interface for both the baseline
         and burdened response.

   The time units of the presented latency MUST be uniform and with
   sufficient precision for the medium or media being tested.

   The latency results for each test SHOULD be reported in the form of a
   table, with a row for each of the tested frame sizes per the
   recommended frame sizes in <a href="#section-3.1.3">section 3.1.3</a>, and SHOULD preserve the
   relationship of latency to ingress/egress interface(s) to assist in
   trending across multiple trials.







<span class="grey">Stopp &amp; Hickman              Informational                     [Page 26]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-27" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


<span class="h3"><a class="selflink" id="section-8.2" href="#section-8.2">8.2</a>.  Forwarding Burdened Group Join Delay</span>

   Objective:

   To determine the time duration it takes a DUT/SUT to start forwarding
   multicast frames from the time a successful IGMP Group Membership
   Report has been issued to the DUT/SUT while forwarding meshed unicast
   traffic.

   Procedure:

   The Forwarding Burdened Group Join Delay test MUST follow the
   described setup for the Group Join Delay test in <a href="#section-6.1">Section 6.1</a>.  In
   addition, another set of test ports MUST be used to burden the
   DUT/SUT (burdening ports).  The burdening ports will be used to
   transmit unicast class traffic to the DUT/SUT in a fully meshed
   traffic pattern as described in <a href="./rfc2285">RFC 2285</a> [<a href="#ref-Ma98" title="&quot;Benchmarking Terminology for LAN Switching Devices&quot;">Ma98</a>].  The DUT/SUT MUST
   learn the appropriate unicast addresses and verified through some
   externally observable method.

   Perform a baseline measurement of Group Join Delay as described in
   <a href="#section-6.1">Section 6.1</a>.  After the baseline measurement is obtained, start
   transmitting the unicast class traffic at a user-specified offered
   load on the set of burdening ports and rerun the Group Join Delay
   test.  The offered load to the ingress port MUST be the same as was
   used in the baseline measurement.

   Reporting Format:

   Similar to <a href="#section-6.1">Section 6.1</a>, the following configuration parameters MUST
   be reflected in the test report:

      o  Frame size(s)
      o  Number of tested egress interfaces on the DUT/SUT
      o  IGMP version
      o  Offered load to ingress interface
      o  Total number of multicast groups
      o  Offered load to burdening ports
      o  Total number of burdening ports
      o  Method used to measure the join delay metric

   The following results MUST be reflected in the test report:

      o  The group join delay time in microseconds per egress
         interface(s) for both the baseline and burdened response.






<span class="grey">Stopp &amp; Hickman              Informational                     [Page 27]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-28" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   The Group Join Delay results for each test MAY be reported in the
   form of a table, with a row for each of the tested frame sizes per
   the recommendations in <a href="#section-3.1.3">section 3.1.3</a>.  Each row or iteration MAY
   specify the group join delay time per egress interface, number of
   frames transmitted and number of frames received for that iteration.

<span class="h2"><a class="selflink" id="section-9" href="#section-9">9</a>.  Security Considerations</span>

   As this document is solely for the purpose of providing metric
   methodology and describes neither a protocol nor a protocol's
   implementation, there are no security considerations associated with
   this document specifically.  Results from these methodologies may
   identify a performance capability or limit of a device or system in a
   particular test context.  However, such results might not be
   representative of the tested entity in an operational network.

<span class="h2"><a class="selflink" id="section-10" href="#section-10">10</a>.  Acknowledgements</span>

   The Benchmarking Methodology Working Group of the IETF and
   particularly Kevin Dubray, Juniper Networks, are to be thanked for
   the many suggestions they collectively made to help complete this
   document.

<span class="h2"><a class="selflink" id="section-11" href="#section-11">11</a>.  Contributions</span>

   The authors would like to acknowledge the following individuals for
   their help and participation of the compilation of this document:
   Hardev Soor, Ixia, and Ralph Daniels, Spirent Communications, both
   who made significant contributions to the earlier versions of this
   document.  In addition, the authors would like to acknowledge the
   members of the task team who helped bring this document to fruition:
   Michele Bustos, Tony De La Rosa, David Newman and Jerry Perser.

<span class="h2"><a class="selflink" id="section-12" href="#section-12">12</a>.  References</span>

<span class="h3"><a class="selflink" id="section-12.1" href="#section-12.1">12.1</a>.  Normative References</span>

   [<a id="ref-Br91">Br91</a>]   Bradner, S., "Benchmarking Terminology for Network
            Interconnection Devices", <a href="./rfc1242">RFC 1242</a>, July 1991.

   [<a id="ref-Br96">Br96</a>]   Bradner, S. and J. McQuaid, "Benchmarking Methodology for
            Network Interconnect Devices", <a href="./rfc2544">RFC 2544</a>, March 1999.

   [<a id="ref-Br97">Br97</a>]   Bradner, S. "Use of Keywords in RFCs to Reflect Requirement
            Levels, <a href="./rfc2119">RFC 2119</a>, March 1997.

   [<a id="ref-Du98">Du98</a>]   Dubray, K., "Terminology for IP Multicast Benchmarking", <a href="./rfc2432">RFC</a>
            <a href="./rfc2432">2432</a>, October 1998.



<span class="grey">Stopp &amp; Hickman              Informational                     [Page 28]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-29" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


   [<a id="ref-IANA1">IANA1</a>]  IANA multicast address assignments,
            <a href="http://www.iana.org/assignments/multicast-addresses">http://www.iana.org/assignments/multicast-addresses</a>

   [<a id="ref-Ma98">Ma98</a>]   Mandeville, R., "Benchmarking Terminology for LAN Switching
            Devices", <a href="./rfc2285">RFC 2285</a>, February 1998.

   [<a id="ref-Me98">Me98</a>]   Meyer, D., "Administratively Scoped IP Multicast", <a href="https://www.rfc-editor.org/bcp/bcp23">BCP 23</a>,
            <a href="./rfc2365">RFC 2365</a>, July 1998.

<span class="h3"><a class="selflink" id="section-12.2" href="#section-12.2">12.2</a>.  Informative References</span>

   [<a id="ref-Ca02">Ca02</a>]   Cain, B., Deering, S., Kouvelas, I., Fenner, B., and A.
            Thyagarajan, "Internet Group Management Protocol, Version
            3", <a href="./rfc3376">RFC 3376</a>, October 2002.

   [<a id="ref-De89">De89</a>]   Deering, S., "Host Extensions for IP Multicasting", STD 5,
            <a href="./rfc1112">RFC 1112</a>, August 1989.

   [<a id="ref-Fe97">Fe97</a>]   Fenner, W., "Internet Group Management Protocol, Version 2",
            <a href="./rfc2236">RFC 2236</a>, November 1997.

   [<a id="ref-Hu95">Hu95</a>]   Huitema, C., "Routing in the Internet", Prentice-Hall, 1995.

   [<a id="ref-Ka98">Ka98</a>]   Kosiur, D., "IP Multicasting: the Complete Guide to
            Interactive Corporate Networks", John Wiley &amp; Sons Inc.,
            1998.

   [<a id="ref-Mt98">Mt98</a>]   Maufer, T., "Deploying IP Multicast in the Enterprise",
            Prentice-Hall, 1998.






















<span class="grey">Stopp &amp; Hickman              Informational                     [Page 29]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-30" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


<span class="h2"><a class="selflink" id="section-13" href="#section-13">13</a>.  Authors' Addresses</span>

   Debra Stopp
   Ixia
   26601 W. Agoura Rd.
   Calabasas, CA  91302
   USA

   Phone: + 1 818 871 1800
   EMail: debby@ixiacom.com


   Brooks Hickman
   Spirent Communications
   26750 Agoura Rd.
   Calabasas, CA  91302
   USA

   Phone: + 1 818 676 2412
   EMail: brooks.hickman@spirentcom.com































<span class="grey">Stopp &amp; Hickman              Informational                     [Page 30]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-31" ></span>
<span class="grey"><a href="./rfc3918">RFC 3918</a>       Methodology for IP Multicast Benchmarking   October 2004</span>


<span class="h2"><a class="selflink" id="section-14" href="#section-14">14</a>.  Full Copyright Statement</span>

   Copyright (C) The Internet Society (2004).

   This document is subject to the rights, licenses and restrictions
   contained in <a href="https://www.rfc-editor.org/bcp/bcp78">BCP 78</a>, and except as set forth therein, the authors
   retain all their rights.

   This document and the information contained herein are provided on an
   "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/S HE
   REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE
   INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR
   IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
   THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed to
   pertain to the implementation or use of the technology described in
   this document or the extent to which any license under such rights
   might or might not be available; nor does it represent that it has
   made any independent effort to identify any such rights.  Information
   on the IETF's procedures with respect to rights in IETF Documents can
   be found in <a href="https://www.rfc-editor.org/bcp/bcp78">BCP 78</a> and <a href="https://www.rfc-editor.org/bcp/bcp79">BCP 79</a>.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use of
   such proprietary rights by implementers or users of this
   specification can be obtained from the IETF on-line IPR repository at
   <a href="http://www.ietf.org/ipr">http://www.ietf.org/ipr</a>.

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at ietf-
   ipr@ietf.org.

Acknowledgement

   Funding for the RFC Editor function is currently provided by the
   Internet Society.







Stopp &amp; Hickman              Informational                     [Page 31]
</pre>