<pre>Network Working Group                                         B. Hickman
Request for Comments: 3511                        Spirent Communications
Category: Informational                                        D. Newman
                                                            Network Test
                                                             S. Tadjudin
                                                  Spirent Communications
                                                               T. Martin
                                                     GVNW Consulting Inc
                                                              April 2003


           <span class="h1">Benchmarking Methodology for Firewall Performance</span>

Status of this Memo

   This memo provides information for the Internet community.  It does
   not specify an Internet standard of any kind.  Distribution of this
   memo is unlimited.

Copyright Notice

   Copyright (C) The Internet Society (2003).  All Rights Reserved.

Abstract

   This document discusses and defines a number of tests that may be
   used to describe the performance characteristics of firewalls.  In
   addition to defining the tests, this document also describes specific
   formats for reporting the results of the tests.

   This document is a product of the Benchmarking Methodology Working
   Group (BMWG) of the Internet Engineering Task Force (IETF).

Table of Contents

   <a href="#section-1">1</a>. Introduction . . . . . . . . . . . . . . . . . . . . . . . .  <a href="#page-2">2</a>
   <a href="#section-2">2</a>. Requirements . . . . . . . . . . . . . . . . . . . . . . . .  <a href="#page-2">2</a>
   <a href="#section-3">3</a>. Scope  . . . . . . . . . . . . . . . . . . . . . . . . . . .  <a href="#page-3">3</a>
   <a href="#section-4">4</a>. Test setup . . . . . . . . . . . . . . . . . . . . . . . . .  <a href="#page-3">3</a>
      <a href="#section-4.1">4.1</a> Test Considerations. . . . . . . . . . . . . . . . . . .  <a href="#page-4">4</a>
      <a href="#section-4.2">4.2</a> Virtual Client/Servers . . . . . . . . . . . . . . . . .  <a href="#page-4">4</a>
      <a href="#section-4.3">4.3</a> Test Traffic Requirements. . . . . . . . . . . . . . . .  <a href="#page-5">5</a>
      <a href="#section-4.4">4.4</a> DUT/SUT Traffic Flows. . . . . . . . . . . . . . . . . .  <a href="#page-5">5</a>
      <a href="#section-4.5">4.5</a> Multiple Client/Server Testing . . . . . . . . . . . . .  <a href="#page-5">5</a>
      <a href="#section-4.6">4.6</a> Network Address Translation (NAT). . . . . . . . . . . .  <a href="#page-6">6</a>
      <a href="#section-4.7">4.7</a> Rule Sets. . . . . . . . . . . . . . . . . . . . . . . .  <a href="#page-6">6</a>
      <a href="#section-4.8">4.8</a> Web Caching. . . . . . . . . . . . . . . . . . . . . . .  <a href="#page-6">6</a>
      <a href="#section-4.9">4.9</a> Authentication . . . . . . . . . . . . . . . . . . . . .  <a href="#page-7">7</a>



<span class="grey">Hickman, et al.              Informational                      [Page 1]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-2" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


      <a href="#section-4.10">4.10</a> TCP Stack Considerations. . . . . . . . . . . . . . . .  <a href="#page-7">7</a>
   <a href="#section-5">5</a>. Benchmarking Tests . . . . . . . . . . . . . . . . . . . . .  <a href="#page-7">7</a>
      <a href="#section-5.1">5.1</a> IP throughput. . . . . . . . . . . . . . . . . . . . . .  <a href="#page-7">7</a>
      <a href="#section-5.2">5.2</a> Concurrent TCP Connection Capacity . . . . . . . . . . .  <a href="#page-9">9</a>
      <a href="#section-5.3">5.3</a> Maximum TCP Connection Establishment Rate. . . . . . . . <a href="#page-12">12</a>
      <a href="#section-5.4">5.4</a> Maximum TCP Connection Tear Down Rate. . . . . . . . . . <a href="#page-14">14</a>
      <a href="#section-5.5">5.5</a> Denial Of Service Handling . . . . . . . . . . . . . . . <a href="#page-16">16</a>
      <a href="#section-5.6">5.6</a> HTTP Transfer Rate . . . . . . . . . . . . . . . . . . . <a href="#page-18">18</a>
      <a href="#section-5.7">5.7</a> Maximum HTTP Transaction Rate. . . . . . . . . . . . . . <a href="#page-21">21</a>
      <a href="#section-5.8">5.8</a> Illegal Traffic Handling . . . . . . . . . . . . . . . . <a href="#page-23">23</a>
      <a href="#section-5.9">5.9</a> IP Fragmentation Handling. . . . . . . . . . . . . . . . <a href="#page-24">24</a>
      <a href="#section-5.10">5.10</a> Latency . . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-26">26</a>
   <a href="#section-6">6</a>. References . . . . . . . . . . . . . . . . . . . . . . . . . <a href="#page-29">29</a>
      <a href="#section-6.1">6.1</a> Normative References . . . . . . . . . . . . . . . . . . <a href="#page-29">29</a>
      <a href="#section-6.2">6.2</a> Informative References . . . . . . . . . . . . . . . . . <a href="#page-30">30</a>
   <a href="#section-7">7</a>. Security Consideration . . . . . . . . . . . . . . . . . . . <a href="#page-30">30</a>
   <a href="#appendix-A">Appendix A</a> - HyperText Transfer Protocol (HTTP) . . . . . . . . <a href="#page-31">31</a>
   <a href="#appendix-B">Appendix B</a> - Connection Establishment Time Measurements . . . . <a href="#page-31">31</a>
   <a href="#appendix-C">Appendix C</a> - Connection Tear Down Time Measurements . . . . . . <a href="#page-32">32</a>
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . <a href="#page-33">33</a>
   Full Copyright Statement  . . . . . . . . . . . . . . . . . . . <a href="#page-34">34</a>

<span class="h2"><a class="selflink" id="section-1" href="#section-1">1</a>. Introduction</span>

   This document provides methodologies for the performance benchmarking
   of firewalls.  It covers four areas: forwarding, connection, latency
   and filtering.  In addition to defining tests, this document also
   describes specific formats for reporting test results.

   A previous document, "Benchmarking Terminology for Firewall
   Performance" [<a href="#ref-1" title="&quot;Benchmarking Terminology for Firewall Devices&quot;">1</a>], defines many of the terms that are used in this
   document.  The terminology document SHOULD be consulted before
   attempting to make use of this document.

<span class="h2"><a class="selflink" id="section-2" href="#section-2">2</a>. Requirements</span>

   In this document, the words that are used to define the significance
   of each particular requirement are capitalized.  These words are:

   *  "MUST" This word, or the words "REQUIRED" and "SHALL" mean that
      the item is an absolute requirement of the specification.

   *  "SHOULD" This word or the adjective "RECOMMENDED" means that there
      may exist valid reasons in particular circumstances to ignore this
      item, but the full implications should be understood and the case
      carefully weighed before choosing a different course.





<span class="grey">Hickman, et al.              Informational                      [Page 2]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-3" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   *  "MAY" This word or the adjective "OPTIONAL" means that this item
      is truly optional.  One vendor may choose to include the item
      because a particular marketplace requires it or because it
      enhances the product, for example; another vendor may omit the
      same item.

   An implementation is not compliant if it fails to satisfy one or more
   of the MUST requirements.  An implementation that satisfies all the
   MUST and all the SHOULD requirements is said to be "unconditionally
   compliant"; one that satisfies all the MUST requirements but not all
   the SHOULD requirements is said to be "conditionally compliant".

<span class="h2"><a class="selflink" id="section-3" href="#section-3">3</a>. Scope</span>

   Firewalls can control access between networks.  Usually, a firewall
   protects a private network from public or shared network(s) to which
   it is connected.  A firewall can be as simple as a single device that
   filters packets or as complex as a group of devices that combine
   packet filtering and application-level proxy and network translation
   services.  This document focuses on benchmarking firewall
   performance, wherever possible, independent of implementation.

<span class="h2"><a class="selflink" id="section-4" href="#section-4">4</a>. Test Setup</span>

   Test configurations defined in this document will be confined to
   dual-homed and tri-homed as shown in figure 1 and figure 2
   respectively.

   Firewalls employing dual-homed configurations connect two networks.
   One interface of the firewall is attached to the unprotected network
   [<a href="#ref-1" title="&quot;Benchmarking Terminology for Firewall Devices&quot;">1</a>], typically the public network (Internet).  The other interface is
   connected to the protected network [<a href="#ref-1" title="&quot;Benchmarking Terminology for Firewall Devices&quot;">1</a>], typically the internal LAN.

   In the case of dual-homed configurations, servers which are made
   accessible to the public (Unprotected) network are attached to the
   private (Protected) network.















<span class="grey">Hickman, et al.              Informational                      [Page 3]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-4" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   +----------+                                       +----------+
   |          |    |       +----------+        |      |          |
   | Servers/ |----|       |          |        |------| Servers/ |
   | Clients  |    |       |          |        |      | Clients  |
   |          |    |-------|  DUT/SUT |--------|      |          |
   +----------+    |       |          |        |      +----------+
        Protected  |       +----------+        | Unprotected
         Network   |                           |   Network
                       Figure 1 (Dual-Homed)

   Tri-homed [<a href="#ref-1" title="&quot;Benchmarking Terminology for Firewall Devices&quot;">1</a>] configurations employ a third segment called a
   Demilitarized Zone (DMZ).  With tri-homed configurations, servers
   accessible to the public network are attached to the DMZ.  Tri-Homed
   configurations offer additional security by separating server(s)
   accessible to the public network from internal hosts.

   +----------+                                       +----------+
   |          |    |       +----------+        |      |          |
   | Clients  |----|       |          |        |------| Servers/ |
   |          |    |       |          |        |      | Clients  |
   +----------+    |-------|  DUT/SUT |--------|      |          |
                   |       |          |        |      +----------+
                   |       +----------+        |
         Protected |            |              | Unprotected
          Network               |                   Network
                                |
                          -----------------
                                    |    DMZ
                                    |
                                    |
                             +-----------+
                             |           |
                             | Servers   |
                             |           |
                             +-----------+

                          Figure 2 (Tri-Homed)

<span class="h3"><a class="selflink" id="section-4.1" href="#section-4.1">4.1</a> Test Considerations</span>

<span class="h3"><a class="selflink" id="section-4.2" href="#section-4.2">4.2</a> Virtual Clients/Servers</span>

   Since firewall testing may involve data sources which emulate
   multiple users or hosts, the methodology uses the terms virtual
   clients/servers.  For these firewall tests, virtual clients/servers
   specify application layer entities which may not be associated with a
   unique physical interface.  For example, four virtual clients may




<span class="grey">Hickman, et al.              Informational                      [Page 4]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-5" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   originate from the same data source [<a href="#ref-1" title="&quot;Benchmarking Terminology for Firewall Devices&quot;">1</a>].  The test report MUST
   indicate the number of virtual clients and virtual servers
   participating in the test.

<span class="h3"><a class="selflink" id="section-4.3" href="#section-4.3">4.3</a> Test Traffic Requirements</span>

   While the function of a firewall is to enforce access control
   policies, the criteria by which those policies are defined vary
   depending on the implementation.  Firewalls may use network layer,
   transport layer or, in many cases, application-layer criteria to make
   access-control decisions.

   For the purposes of benchmarking firewall performance, this document
   references HTTP 1.1 or higher as the application layer entity.  The
   methodologies MAY be used as a template for benchmarking with other
   applications.  Since testing may involve proxy based DUT/SUTs, HTTP
   version considerations are discussed in <a href="#appendix-A">appendix A</a>.

<span class="h3"><a class="selflink" id="section-4.4" href="#section-4.4">4.4</a> DUT/SUT Traffic Flows</span>

   Since the number of interfaces are not fixed, the traffic flows will
   be dependent upon the configuration used in benchmarking the DUT/SUT.
   Note that the term "traffic flows" is associated with client-to-
   server requests.

   For Dual-Homed configurations, there are two unique traffic flows:

      Client         Server
      ------         ------
      Protected   -&gt; Unprotected
      Unprotected -&gt; Protected

   For Tri-Homed configurations, there are three unique traffic flows:

      Client         Server
      ------         ------
      Protected -&gt;   Unprotected
      Protected -&gt;   DMZ
      Unprotected -&gt; DMZ

<span class="h3"><a class="selflink" id="section-4.5" href="#section-4.5">4.5</a> Multiple Client/Server Testing</span>

   One or more clients may target multiple servers for a given
   application.  Each virtual client MUST initiate connections in a
   round-robin fashion.  For example, if the test consisted of six
   virtual clients targeting three servers, the pattern would be as
   follows:




<span class="grey">Hickman, et al.              Informational                      [Page 5]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-6" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


      Client          Target Server (In order of request)
      #1              1     2     3     1...
      #2              2     3     1     2...
      #3              3     1     2     3...
      #4              1     2     3     1...
      #5              2     3     1     2...
      #6              3     1     2     3...

<span class="h3"><a class="selflink" id="section-4.6" href="#section-4.6">4.6</a> Network Address Translation (NAT)</span>

   Many firewalls implement network address translation (NAT) [<a href="#ref-1" title="&quot;Benchmarking Terminology for Firewall Devices&quot;">1</a>], a
   function which translates private internet addresses to public
   internet addresses.  This involves additional processing on the part
   of the DUT/SUT and may impact performance.  Therefore, tests SHOULD
   be ran with NAT disabled and NAT enabled to determine the performance
   differential, if any.  The test report MUST indicate whether NAT was
   enabled or disabled.

<span class="h3"><a class="selflink" id="section-4.7" href="#section-4.7">4.7</a> Rule Sets</span>

   Rule sets [<a href="#ref-1" title="&quot;Benchmarking Terminology for Firewall Devices&quot;">1</a>] are a collection of access control policies that
   determine which packets the DUT/SUT will forward and which it will
   reject [<a href="#ref-1" title="&quot;Benchmarking Terminology for Firewall Devices&quot;">1</a>].  Since criteria by which these access control policies
   may be defined will vary depending on the capabilities of the
   DUT/SUT, the following is limited to providing guidelines for
   configuring rule sets when benchmarking the performance of the
   DUT/SUT.

   It is RECOMMENDED that a rule be entered for each host (Virtual
   client).  In addition, testing SHOULD be performed using different
   size rule sets to determine its impact on the performance of the
   DUT/SUT.  Rule sets MUST be configured in a manner, such that, rules
   associated with actual test traffic are configured at the end of the
   rule set and not at the beginning.

   The DUT/SUT SHOULD be configured to deny access to all traffic which
   was not previously defined in the rule set.  The test report SHOULD
   include the DUT/SUT configured rule set(s).

<span class="h3"><a class="selflink" id="section-4.8" href="#section-4.8">4.8</a> Web Caching</span>

   Some firewalls include caching agents to reduce network load.  When
   making a request through a caching agent, the caching agent attempts
   to service the response from its internal memory.  The cache itself
   saves responses it receives, such as responses for HTTP GET requests.
   Testing SHOULD be performed with any caching agents on the DUT/SUT
   disabled.




<span class="grey">Hickman, et al.              Informational                      [Page 6]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-7" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


<span class="h3"><a class="selflink" id="section-4.9" href="#section-4.9">4.9</a> Authentication</span>

   Access control may involve authentication processes such as user,
   client or session authentication.  Authentication is usually
   performed by devices external to the firewall itself, such as an
   authentication server(s) and may add to the latency of the system.
   Any authentication processes MUST be included as part of connection
   setup process.

<span class="h3"><a class="selflink" id="section-4.10" href="#section-4.10">4.10</a> TCP Stack Considerations</span>

   Some test instruments allow configuration of one or more TCP stack
   parameters, thereby influencing the traffic flows which will be
   offered and impacting performance measurements.  While this document
   does not attempt to specify which TCP parameters should be
   configurable, any such TCP parameter(s) MUST be noted in the test
   report.  In addition, when comparing multiple DUT/SUTs, the same TCP
   parameters MUST be used.

<span class="h2"><a class="selflink" id="section-5" href="#section-5">5</a>. Benchmarking Tests</span>

<span class="h3"><a class="selflink" id="section-5.1" href="#section-5.1">5.1</a> IP Throughput</span>

<span class="h4"><a class="selflink" id="section-5.1.1" href="#section-5.1.1">5.1.1</a> Objective</span>

   To determine the throughput of network-layer data traversing the
   DUT/SUT, as defined in <a href="./rfc1242">RFC 1242</a> [<a href="#ref-3" title="&quot;Benchmarking Terminology for Network Interconnection Devices&quot;">3</a>].  Note that while <a href="./rfc1242">RFC 1242</a> uses
   the term frames, which is associated with the link layer, the
   procedure uses the term packets, since it is referencing the network
   layer.

<span class="h4"><a class="selflink" id="section-5.1.2" href="#section-5.1.2">5.1.2</a> Setup Parameters</span>

   The following parameters MUST be defined:

      Packet size - Number of bytes in the IP packet, exclusive of any
      link layer header or checksums.

      Test Duration - Duration of the test, expressed in seconds.

<span class="h4"><a class="selflink" id="section-5.1.3" href="#section-5.1.3">5.1.3</a> Procedure</span>

   The test instrument MUST offer unicast IP packets to the DUT/SUT at a
   constant rate.  The test MAY consist of either bi-directional or
   unidirectional traffic; for example, an emulated client may offer a
   unicast stream of packets to an emulated server, or the test
   instrument may simulate a client/server exchange by offering
   bidirectional traffic.



<span class="grey">Hickman, et al.              Informational                      [Page 7]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-8" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   This test will employ an iterative search algorithm.  Each iteration
   will involve the test instrument varying the intended load until the
   maximum rate, at which no packet loss occurs, is found.  Since
   backpressure mechanisms may be employed, resulting in the intended
   load and offered load being different, the test SHOULD be performed
   in either a packet based or time based manner as described in <a href="./rfc2889">RFC</a>
   <a href="./rfc2889">2889</a> [<a href="#ref-5" title="&quot;Benchmarking Methodology for LAN Switching Devices&quot;">5</a>].  As with <a href="./rfc1242">RFC 1242</a>, the term packet is used in place of
   frame.  The duration of the test portion of each trial MUST be at
   least 30 seconds.

   It is RECOMMENDED to perform the throughput measurements with
   different packet sizes.  When testing with different packet sizes the
   DUT/SUT configuration MUST remain the same.

<span class="h4"><a class="selflink" id="section-5.1.4" href="#section-5.1.4">5.1.4</a> Measurement</span>

<span class="h5"><a class="selflink" id="section-5.1.4.1" href="#section-5.1.4.1">5.1.4.1</a> Network Layer</span>

   Throughput:
      Maximum offered load, expressed in either bits per second or
      packets per second, at which no packet loss is detected.  The bits
      to be counted are in the IP packet (header plus payload); other
      fields, such as link-layer headers and trailers, MUST NOT be
      included in the measurement.

   Forwarding Rate:
      Forwarding rate, expressed in either bits per second or packets
      per second, the device is observed to successfully forward to the
      correct destination interface in response to a specified offered
      load.  The bits to be counted are in the IP packet (header plus
      payload); other fields, such as link-layer headers and trailers,
      MUST NOT be included in the measurement.

<span class="h4"><a class="selflink" id="section-5.1.5" href="#section-5.1.5">5.1.5</a> Reporting Format</span>

   The test report MUST note the packet size(s), test duration,
   throughput and forwarding rate. In addition, the test report MUST
   conform to the reporting requirements set in <a href="#section-4">section 4</a>, Test Setup.
   If the test involved offering packets which target more than one
   segment (Protected, Unprotected or DMZ), the report MUST identify the
   results as an aggregate throughput measurement.

   The throughput results SHOULD be reported in the format of a table
   with a row for each of the tested packet sizes.  There SHOULD be
   columns for the packet size, the intended load, the offered load,
   resultant throughput and forwarding rate for each test.





<span class="grey">Hickman, et al.              Informational                      [Page 8]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-9" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   The intermediate results of the search algorithm MAY be saved in log
   file which includes the packet size, test duration and for each
   iteration:

      - Step Iteration
      - Pass/Fail Status
      - Total packets offered
      - Total packets forwarded
      - Intended load
      - Offered load (If applicable)
      - Forwarding rate

<span class="h3"><a class="selflink" id="section-5.2" href="#section-5.2">5.2</a> Concurrent TCP Connection Capacity</span>

<span class="h4"><a class="selflink" id="section-5.2.1" href="#section-5.2.1">5.2.1</a> Objective</span>

   To determine the maximum number of concurrent TCP connections
   supported through or with the DUT/SUT, as defined in <a href="./rfc2647">RFC 2647</a> [<a href="#ref-1" title="&quot;Benchmarking Terminology for Firewall Devices&quot;">1</a>].
   This test is intended to find the maximum number of entries the
   DUT/SUT can store in its connection table.

<span class="h4"><a class="selflink" id="section-5.2.2" href="#section-5.2.2">5.2.2</a> Setup Parameters</span>

   The following parameters MUST be defined for all tests:

<span class="h5"><a class="selflink" id="section-5.2.2.1" href="#section-5.2.2.1">5.2.2.1</a> Transport-Layer Setup Parameters</span>

   Connection Attempt Rate:
      The aggregate rate, expressed in connections per second, at which
      TCP connection requests are attempted.  The rate SHOULD be set at
      or lower than the maximum rate at which the DUT/SUT can accept
      connection requests.

   Aging Time:
      The time, expressed in seconds, the DUT/SUT will keep a connection
      in its connection table after receiving a TCP FIN or RST packet.

<span class="h5"><a class="selflink" id="section-5.2.2.2" href="#section-5.2.2.2">5.2.2.2</a> Application-Layer Setup Parameters</span>

   Validation Method:
      HTTP 1.1 or higher MUST be used for this test for both clients and
      servers.  The client and server MUST use the same HTTP version.

   Object Size:
      Defines the number of bytes, excluding any bytes associated with
      the HTTP header, to be transferred in response to an HTTP 1.1 or
      higher GET request.




<span class="grey">Hickman, et al.              Informational                      [Page 9]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-10" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


<span class="h4"><a class="selflink" id="section-5.2.3" href="#section-5.2.3">5.2.3</a> Procedure</span>

   This test will employ an iterative search algorithm to determine the
   maximum number of concurrent TCP connections supported through or
   with the DUT/SUT.

   For each iteration, the aggregate number of concurrent TCP
   connections attempted by the virtual client(s) will be varied.  The
   destination address will be that of the server or that of the NAT
   proxy.  The aggregate rate will be defined by connection attempt
   rate, and will be attempted in a round-robin fashion (See 4.5).

   To validate all connections, the virtual client(s) MUST request an
   object using an HTTP 1.1 or higher GET request.  The requests MUST be
   initiated on each connection after all of the TCP connections have
   been established.

   When testing proxy-based DUT/SUTs, the virtual client(s) MUST request
   two objects using HTTP 1.1 or higher GET requests.  The first GET
   request is required for connection time establishment [<a href="#ref-1" title="&quot;Benchmarking Terminology for Firewall Devices&quot;">1</a>]
   measurements as specified in <a href="#appendix-B">appendix B</a>.  The second request is used
   for validation as previously mentioned.  When comparing proxy and
   non-proxy based DUT/SUTs, the test MUST be performed in the same
   manner.

   Between each iteration, it is RECOMMENDED that the test instrument
   issue a TCP RST referencing each connection attempted for the
   previous iteration, regardless of whether or not the connection
   attempt was successful.  The test instrument will wait for aging time
   before continuing to the next iteration.

<span class="h4"><a class="selflink" id="section-5.2.4" href="#section-5.2.4">5.2.4</a> Measurements</span>

<span class="h5"><a class="selflink" id="section-5.2.4.1" href="#section-5.2.4.1">5.2.4.1</a> Application-Layer measurements</span>

   Number of objects requested

   Number of objects returned

<span class="h5"><a class="selflink" id="section-5.2.4.2" href="#section-5.2.4.2">5.2.4.2</a> Transport-Layer measurements</span>

   Maximum concurrent connections:
      Total number of TCP connections open for the last successful
      iteration performed in the search algorithm.

   Minimum connection establishment time:
      Lowest TCP connection establishment time measured, as defined in
      <a href="#appendix-B">appendix B</a>.



<span class="grey">Hickman, et al.              Informational                     [Page 10]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-11" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   Maximum connection establishment time:
      Highest TCP connection establishment time measured, as defined in
      <a href="#appendix-B">appendix B</a>.

   Average connection establishment time:
      The mean of all measurements of connection establishment times.

   Aggregate connection establishment time:
      The total of all measurements of connection establishment times.

<span class="h4"><a class="selflink" id="section-5.2.5" href="#section-5.2.5">5.2.5</a> Reporting Format</span>

   The test report MUST conform to the reporting requirements set in
   <a href="#section-4">section 4</a>, Test Setup.

<span class="h5"><a class="selflink" id="section-5.2.5.1" href="#section-5.2.5.1">5.2.5.1</a> Application-Layer Reporting:</span>

   The test report MUST note the object size, number of completed
   requests and number of completed responses.

   The intermediate results of the search algorithm MAY be reported in a
   tabular format with a column for each iteration.  There SHOULD be
   rows for the number of requests attempted, number and percentage
   requests completed, number of responses attempted, number and
   percentage of responses completed.  The table MAY be combined with
   the transport-layer reporting, provided that the table identify this
   as an application layer measurement.

   Version information:
      The test report MUST note the version of HTTP client(s) and
      server(s).

<span class="h5"><a class="selflink" id="section-5.2.5.2" href="#section-5.2.5.2">5.2.5.2</a> Transport-Layer Reporting:</span>

   The test report MUST note the connection attempt rate, aging time,
   minimum TCP connection establishment time, maximum TCP connection
   establishment time, average connection establishment time, aggregate
   connection establishment time and maximum concurrent connections
   measured.

   The intermediate results of the search algorithm MAY be reported in
   the format of a table with a column for each iteration.  There SHOULD
   be rows for the total number of TCP connections attempted, number and
   percentage of TCP connections completed, minimum TCP connection
   establishment time, maximum TCP connection establishment time,
   average connection establishment time and the aggregate connection
   establishment time.




<span class="grey">Hickman, et al.              Informational                     [Page 11]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-12" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


<span class="h3"><a class="selflink" id="section-5.3" href="#section-5.3">5.3</a> Maximum TCP Connection Establishment Rate</span>

<span class="h4"><a class="selflink" id="section-5.3.1" href="#section-5.3.1">5.3.1</a> Objective</span>

   To determine the maximum TCP connection establishment rate through or
   with the DUT/SUT, as defined by <a href="./rfc2647">RFC 2647</a> [<a href="#ref-1" title="&quot;Benchmarking Terminology for Firewall Devices&quot;">1</a>].  This test is intended
   to find the maximum rate the DUT/SUT can update its connection table.

<span class="h4"><a class="selflink" id="section-5.3.2" href="#section-5.3.2">5.3.2</a> Setup Parameters</span>

   The following parameters MUST be defined for all tests:

<span class="h5"><a class="selflink" id="section-5.3.2.1" href="#section-5.3.2.1">5.3.2.1</a> Transport-Layer Setup Parameters</span>

   Number of Connections:
      Defines the aggregate number of TCP connections that must be
      established.

   Aging Time:
      The time, expressed in seconds, the DUT/SUT will keep a connection
      in it's state table after receiving a TCP FIN or RST packet.

<span class="h5"><a class="selflink" id="section-5.3.2.2" href="#section-5.3.2.2">5.3.2.2</a> Application-Layer Setup Parameters</span>

   Validation Method:
      HTTP 1.1 or higher MUST be used for this test for both clients and
      servers.  The client and server MUST use the same HTTP version.

   Object Size:
      Defines the number of bytes, excluding any bytes associated with
      the HTTP header, to be transferred in response to an HTTP 1.1 or
      higher GET request.

<span class="h4"><a class="selflink" id="section-5.3.3" href="#section-5.3.3">5.3.3</a> Procedure</span>

   This test will employ an iterative search algorithm to determine the
   maximum rate at which the DUT/SUT can accept TCP connection requests.

   For each iteration, the aggregate rate at which TCP connection
   requests are attempted by the virtual client(s) will be varied.  The
   destination address will be that of the server or that of the NAT
   proxy.  The aggregate number of connections, defined by number of
   connections, will be attempted in a round-robin fashion (See 4.5).

   The same application-layer object transfers required for validation
   and establishment time measurements as described in the concurrent
   TCP connection capacity test MUST be performed.




<span class="grey">Hickman, et al.              Informational                     [Page 12]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-13" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   Between each iteration, it is RECOMMENDED that the test instrument
   issue a TCP RST referencing each connection attempted for the
   previous iteration, regardless of whether or not the connection
   attempt was successful.  The test instrument will wait for aging time
   before continuing to the next iteration.

<span class="h4"><a class="selflink" id="section-5.3.4" href="#section-5.3.4">5.3.4</a> Measurements</span>

<span class="h5"><a class="selflink" id="section-5.3.4.1" href="#section-5.3.4.1">5.3.4.1</a> Application-Layer measurements</span>

   Number of objects requested

   Number of objects returned

<span class="h5"><a class="selflink" id="section-5.3.4.2" href="#section-5.3.4.2">5.3.4.2</a> Transport-Layer measurements</span>

   Highest connection rate:
      Highest rate, in connections per second, for which all connections
      successfully opened in the search algorithm.

   Minimum connection establishment time:
      Lowest TCP connection establishment time measured, as defined in
      <a href="#appendix-B">appendix B</a>.

   Maximum connection establishment time:
      Highest TCP connection establishment time measured, as defined in
      <a href="#appendix-B">appendix B</a>.

   Average connection establishment time:
      The mean of all measurements of connection establishment times.

   Aggregate connection establishment time:
      The total of all measurements of connection establishment times.

<span class="h4"><a class="selflink" id="section-5.3.5" href="#section-5.3.5">5.3.5</a> Reporting Format</span>

   The test report MUST conform to the reporting requirements set in
   <a href="#section-4">section 4</a>, Test Setup.

<span class="h5"><a class="selflink" id="section-5.3.5.1" href="#section-5.3.5.1">5.3.5.1</a> Application-Layer Reporting:</span>

   The test report MUST note object size(s), number of completed
   requests and number of completed responses.

   The intermediate results of the search algorithm MAY be reported in a
   tabular format with a column for each iteration.  There SHOULD be
   rows for the number of requests attempted, number and percentage
   requests completed, number of responses attempted, number and



<span class="grey">Hickman, et al.              Informational                     [Page 13]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-14" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   percentage of responses completed.  The table MAY be combined with
   the transport-layer reporting, provided that the table identify this
   as an application layer measurement.

   Version information:
      The test report MUST note the version of HTTP client(s) and
      server(s).

<span class="h5"><a class="selflink" id="section-5.3.5.2" href="#section-5.3.5.2">5.3.5.2</a> Transport-Layer Reporting:</span>

   The test report MUST note the number of connections, aging time,
   minimum TCP connection establishment time, maximum TCP connection
   establishment time, average connection establishment time, aggregate
   connection establishment time and highest connection rate measured.

   The intermediate results of the search algorithm MAY be reported in
   the format of a table with a column for each iteration.  There SHOULD
   be rows for the connection attempt rate, total number of TCP
   connections attempted, total number of TCP connections completed,
   minimum TCP connection establishment time, maximum TCP connection
   establishment time, average connection establishment time and the
   aggregate connection establishment time.

<span class="h3"><a class="selflink" id="section-5.4" href="#section-5.4">5.4</a> Maximum TCP Connection Tear Down Rate</span>

<span class="h4"><a class="selflink" id="section-5.4.1" href="#section-5.4.1">5.4.1</a> Objective</span>

   To determine the maximum TCP connection tear down rate through or
   with the DUT/SUT, as defined by <a href="./rfc2647">RFC 2647</a> [<a href="#ref-1" title="&quot;Benchmarking Terminology for Firewall Devices&quot;">1</a>].

<span class="h4"><a class="selflink" id="section-5.4.2" href="#section-5.4.2">5.4.2</a> Setup Parameters</span>

   Number of Connections:
      Defines the number of TCP connections that will be attempted to be
      torn down.

   Aging Time:
      The time, expressed in seconds, the DUT/SUT will keep a connection
      in it's state table after receiving a TCP FIN or RST packet.

   Close Method:
      Defines method for closing TCP connections.  The test MUST be
      performed with either a three-way or four-way handshake.  In a
      four-way handshake, each side sends separate FIN and ACK messages.
      In a three-way handshake, one side sends a combined FIN/ACK
      message upon receipt of a FIN.





<span class="grey">Hickman, et al.              Informational                     [Page 14]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-15" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   Close Direction:
      Defines whether closing of connections are to be initiated from
      the client or from the server.

<span class="h4"><a class="selflink" id="section-5.4.3" href="#section-5.4.3">5.4.3</a> Procedure</span>

   This test will employ an iterative search algorithm to determine the
   maximum TCP connection tear down rate supported by the DUT/SUT.  The
   test iterates through different TCP connection tear down rates with a
   fixed number of TCP connections.

   In the case of proxy based DUT/SUTs, the DUT/SUT will itself receive
   the ACK in response to issuing a FIN packet to close its side of the
   TCP connection.  For validation purposes, the virtual client or
   server, whichever is applicable, MAY verify that the DUT/SUT received
   the final ACK by re-transmitting the final ACK.  A TCP RST should be
   received in response to the retransmitted ACK.

   Between each iteration, it is RECOMMENDED that the virtual client(s)
   or server(s), whichever is applicable, issue a TCP RST referencing
   each connection which was attempted to be torn down, regardless of
   whether or not the connection tear down attempt was successful.  The
   test will wait for aging time before continuing to the next
   iteration.

<span class="h4"><a class="selflink" id="section-5.4.4" href="#section-5.4.4">5.4.4</a> Measurements</span>

   Highest connection tear down rate:
      Highest rate, in connections per second, for which all TCP
      connections were successfully torn down in the search algorithm.

   The following tear down time [<a href="#ref-1" title="&quot;Benchmarking Terminology for Firewall Devices&quot;">1</a>] measurements MUST only include
   connections for which both sides of the connection were successfully
   torn down.  For example, tear down times for connections which are
   left in a FINWAIT-2 [<a href="#ref-8" title="&quot;Transmission Control Protocol&quot;">8</a>] state should not be included:

   Minimum connection tear down time:
      Lowest TCP connection tear down time measured as defined in
      <a href="#appendix-C">appendix C</a>.

   Maximum connection tear down time:
      Highest TCP connection tear down time measured as defined in
      <a href="#appendix-C">appendix C</a>.

   Average connection tear down time:
      The mean of all measurements of connection tear down times.





<span class="grey">Hickman, et al.              Informational                     [Page 15]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-16" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   Aggregate connection tear down time:
      The total of all measurements of connection tear down times.

<span class="h4"><a class="selflink" id="section-5.4.5" href="#section-5.4.5">5.4.5</a> Reporting Format</span>

   The test report MUST note the number of connections, aging time,
   close method, close direction, minimum TCP connection tear down time,
   maximum TCP connection tear down time, average TCP connection tear
   down time and the aggregate TCP connection tear down time and highest
   connection tear down rate measured. In addition, the test report MUST
   conform to the reporting requirements set in <a href="#section-4">section 4</a>, Test Setup.

   The intermediate results of the search algorithm MAY be reported in
   the format of a table with a column for each iteration.  There SHOULD
   be rows for the number of TCP tear downs attempted, number and
   percentage of TCP connection tear downs completed, minimum TCP
   connection tear down time, maximum TCP connection tear down time,
   average TCP connection tear down time, aggregate TCP connection tear
   down time and validation failures, if required.

<span class="h3"><a class="selflink" id="section-5.5" href="#section-5.5">5.5</a> Denial Of Service Handling</span>

<span class="h4"><a class="selflink" id="section-5.5.1" href="#section-5.5.1">5.5.1</a> Objective</span>

   To determine the effect of a denial of service attack on a DUT/SUT
   TCP connection establishment and/or HTTP transfer rates.  The denial
   of service handling test MUST be run after obtaining baseline
   measurements from sections <a href="#section-5.3">5.3</a> and/or 5.6.

   The TCP SYN flood attack exploits TCP's three-way handshake mechanism
   by having an attacking source host generate TCP SYN packets with
   random source addresses towards a victim host, thereby consuming that
   host's resources.

<span class="h4"><a class="selflink" id="section-5.5.2" href="#section-5.5.2">5.5.2</a> Setup Parameters</span>

   Use the same setup parameters as defined in <a href="#section-5.3.2">section 5.3.2</a> or 5.6.2,
   depending on whether testing against the baseline TCP connection
   establishment rate test or HTTP transfer rate test, respectfully.

   In addition, the following setup parameters MUST be defined:

   SYN attack rate:
      Rate, expressed in packets per second, at which the server(s) or
      NAT proxy address is targeted with TCP SYN packets.






<span class="grey">Hickman, et al.              Informational                     [Page 16]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-17" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


<span class="h4"><a class="selflink" id="section-5.5.3" href="#section-5.5.3">5.5.3</a> Procedure</span>

   Use the same procedure as defined in <a href="#section-5.3.3">section 5.3.3</a> or 5.6.3,
   depending on whether testing against the baseline TCP connection
   establishment rate or HTTP transfer rate test, respectfully.  In
   addition, the test instrument will generate TCP SYN packets targeting
   the server(s) IP address or NAT proxy address at a rate defined by
   SYN attack rate.

   The test instrument originating the TCP SYN attack MUST be attached
   to the unprotected network.  In addition, the test instrument MUST
   not respond to the SYN/ACK packets sent by target server or NAT proxy
   in response to the SYN packet.

   Some firewalls employ mechanisms to guard against SYN attacks.  If
   such mechanisms exist on the DUT/SUT, tests SHOULD be run with these
   mechanisms enabled and disabled to determine how well the DUT/SUT can
   maintain, under such attacks, the baseline connection establishment
   rates and HTTP transfer rates determined in <a href="#section-5.3">section 5.3</a> and <a href="#section-5.6">section</a>
   <a href="#section-5.6">5.6</a>, respectively.

<span class="h4"><a class="selflink" id="section-5.5.4" href="#section-5.5.4">5.5.4</a> Measurements</span>

   Perform the same measurements as defined in <a href="#section-5.3.4">section 5.3.4</a> or 5.6.4,
   depending on whether testing against the baseline TCP connection
   establishment rate test or HTTP transfer rate, respectfully.

   In addition, the test instrument SHOULD track TCP SYN packets
   associated with the SYN attack which the DUT/SUT forwards on the
   protected or DMZ interface(s).

<span class="h4"><a class="selflink" id="section-5.5.5" href="#section-5.5.5">5.5.5</a> Reporting Format</span>

   The test SHOULD use the same reporting format as described in <a href="#section-5.3.5">section</a>
   <a href="#section-5.3.5">5.3.5</a> or 5.6.5, depending on whether testing against the baseline TCP
   connection establishment rate test or HTTP transfer rate,
   respectfully.

   In addition, the report MUST indicate a denial of service handling
   test, SYN attack rate, number of TCP SYN attack packets transmitted
   and the number of TCP SYN attack packets forwarded by the DUT/SUT.
   The report MUST indicate whether or not the DUT has any SYN attack
   mechanisms enabled.








<span class="grey">Hickman, et al.              Informational                     [Page 17]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-18" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


<span class="h3"><a class="selflink" id="section-5.6" href="#section-5.6">5.6</a> HTTP Transfer Rate</span>

<span class="h4"><a class="selflink" id="section-5.6.1" href="#section-5.6.1">5.6.1</a> Objective</span>

   To determine the transfer rate of HTTP requested object traversing
   the DUT/SUT.

<span class="h4"><a class="selflink" id="section-5.6.2" href="#section-5.6.2">5.6.2</a> Setup Parameters</span>

   The following parameters MUST be defined for all tests:

<span class="h5"><a class="selflink" id="section-5.6.2.1" href="#section-5.6.2.1">5.6.2.1</a> Transport-Layer Setup Parameters</span>

   Number of connections:
      Defines the aggregate number of connections attempted.  The number
      SHOULD be a multiple of the number of virtual clients
      participating in the test.

   Close Method:
      Defines the method for closing TCP connections.  The test MUST be
      performed with either a three-way or four-way handshake.  In a
      four-way handshake, each side sends separate FIN and ACK messages.
      In a three-way handshake, one side sends a combined FIN/ACK
      message upon receipt of a FIN.

   Close Direction:
      Defines whether closing of connections are to be initiated from
      the client or from the server.

<span class="h5"><a class="selflink" id="section-5.6.2.2" href="#section-5.6.2.2">5.6.2.2</a> Application-Layer Setup Parameters</span>

   Session Type:
      The virtual clients/servers MUST use HTTP 1.1 or higher.  The
      client and server MUST use the same HTTP version.

   GET requests per connection:
      Defines the number of HTTP 1.1 or higher GET requests attempted
      per connection.

   Object Size:
      Defines the number of bytes, excluding any bytes associated with
      the HTTP header, to be transferred in response to an HTTP 1.1 or
      higher GET request.








<span class="grey">Hickman, et al.              Informational                     [Page 18]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-19" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


<span class="h4"><a class="selflink" id="section-5.6.3" href="#section-5.6.3">5.6.3</a> Procedure</span>

   Each HTTP 1.1 or higher virtual client will request one or more
   objects from an HTTP 1.1 or higher server using one or more HTTP GET
   requests over each connection.  The aggregate number of connections
   attempted, defined by number of connections, MUST be evenly divided
   among all of the participating virtual clients.

   If the virtual client(s) make multiple HTTP GET requests per
   connection, it MUST request the same object size for each GET
   request.  Multiple iterations of this test may be run with objects of
   different sizes.

<span class="h4"><a class="selflink" id="section-5.6.4" href="#section-5.6.4">5.6.4</a> Measurements</span>

<span class="h5"><a class="selflink" id="section-5.6.4.1" href="#section-5.6.4.1">5.6.4.1</a> Application-Layer measurements</span>

   Average Transfer Rate :
      The average transfer rate of the DUT/SUT MUST be measured and
      shall be referenced to the requested object(s).  The measurement
      will start on transmission of the first bit of the first requested
      object and end on transmission of the last bit of the last
      requested object.  The average transfer rate, in bits per second,
      will be calculated using the following formula:

                             OBJECTS * OBJECTSIZE * 8
   TRANSFER RATE (bit/s) =  --------------------------
                                    DURATION

   OBJECTS    - Total number of objects successfully transferred across
                all connections.

   OBJECTSIZE - Object size in bytes

   DURATION   - Aggregate transfer time based on aforementioned time
                references.

<span class="h5"><a class="selflink" id="section-5.6.4.2" href="#section-5.6.4.2">5.6.4.2</a> Measurements at or below the Transport-Layer</span>

   The following measurements SHOULD be performed for each connection-
   oriented protocol:

   Goodput [<a href="#ref-1" title="&quot;Benchmarking Terminology for Firewall Devices&quot;">1</a>]:
      Goodput as defined in <a href="./rfc2647#section-3.17">section&nbsp;3.17 of RFC 2647</a>.  Measurements MUST
      only reference the protocol payload, excluding any of the protocol
      header.  In addition, the test instrument MUST exclude any bits
      associated with the connection establishment, connection tear
      down, security associations [<a href="#ref-1" title="&quot;Benchmarking Terminology for Firewall Devices&quot;">1</a>] or connection maintenance [<a href="#ref-1" title="&quot;Benchmarking Terminology for Firewall Devices&quot;">1</a>].



<span class="grey">Hickman, et al.              Informational                     [Page 19]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-20" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


      Since connection-oriented protocols require that data be
      acknowledged, the offered load [<a href="#ref-4" title="&quot;Benchmarking Terminology for LAN Switching Devices&quot;">4</a>] will be varying.  Therefore,
      the test instrument should measure the average forwarding rate
      over the duration of the test.  Measurement should start on
      transmission of the first bit of the payload of the first datagram
      and end on transmission of the last bit of the payload of the last
      datagram.

   Number of bytes transferred - Total payload bytes transferred.

   Number of Timeouts - Total number of timeout events.

   Retransmitted bytes - Total number of retransmitted bytes.

<span class="h4"><a class="selflink" id="section-5.6.5" href="#section-5.6.5">5.6.5</a> Reporting Format</span>

   The test report MUST conform to the reporting requirements set in
   <a href="#section-4">section 4</a>, Test Setup.

<span class="h5"><a class="selflink" id="section-5.6.5.1" href="#section-5.6.5.1">5.6.5.1</a> Application-Layer reporting</span>

   The test report MUST note number of GET requests per connection and
   object size(s).

   The transfer rate results SHOULD be reported in tabular form with a
   column for each of the object sizes tested.  There SHOULD be a row
   for the number and percentage of completed requests, number and
   percentage of completed responses, and the resultant transfer rate
   for each iteration of the test.

   Failure analysis:
      The test report SHOULD indicate the number and percentage of HTTP
      GET request and responses that failed to complete.

   Version information:
      The test report MUST note the version of HTTP client(s) and
      server(s).

<span class="h5"><a class="selflink" id="section-5.6.5.2" href="#section-5.6.5.2">5.6.5.2</a> Transport-Layer and below reporting</span>

   The test report MUST note the number of connections, close method,
   close direction and the protocol for which the measurement was made.

   The results SHOULD be reported in tabular form for each of the HTTP
   object sizes tested.  There SHOULD be a row for the total bytes
   transferred, total timeouts, total retransmitted bytes and and
   resultant goodput.  Note that total bytes refers to total datagram
   payload bytes transferred.  The table MAY be combined with the



<span class="grey">Hickman, et al.              Informational                     [Page 20]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-21" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   application layer reporting, provided the table clearly identifies
   the protocol for which the measurement was made.

   Failure analysis:
      The test report SHOULD indicate the number and percentage of
      connection establishment failures as well as number and percentage
      of TCP tear down failures.

   It is RECOMMENDED that the report include a graph to plot the
   distribution of both connection establishment failures and connection
   tear down failures.  The x coordinate SHOULD be the elapsed test
   time, the y coordinate SHOULD be the number of failures for a given
   sampling period.  There SHOULD be two lines on the graph, one for
   connection failures and one for tear down failures.  The graph MUST
   note the sampling period.

<span class="h3"><a class="selflink" id="section-5.7" href="#section-5.7">5.7</a> Maximum HTTP Transaction Rate</span>

<span class="h4"><a class="selflink" id="section-5.7.1" href="#section-5.7.1">5.7.1</a> Objective</span>

   Determine the maximum transaction rate the DUT/SUT can sustain.  This
   test is intended to find the maximum rate at which users can access
   objects.

<span class="h4"><a class="selflink" id="section-5.7.2" href="#section-5.7.2">5.7.2</a> Setup Parameters</span>

<span class="h5"><a class="selflink" id="section-5.7.2.1" href="#section-5.7.2.1">5.7.2.1</a> Transport-Layer Setup Parameters</span>

   Close Method:
      Defines method for closing TCP connections.  The test MUST be
      performed with either a three-way or four-way handshake.  In a
      four-way handshake, each side sends separate FIN and ACK messages.
      In a three-way handshake, one side sends a combined FIN/ACK
      message upon receipt of a FIN.

   Close Direction:
      Defines whether closing of connections are to be initiated from
      the client or from the server.

<span class="h5"><a class="selflink" id="section-5.7.2.2" href="#section-5.7.2.2">5.7.2.2</a> Application-Layer Setup Parameters</span>

   Session Type:
      HTTP 1.1 or higher MUST be used for this test.  The client and
      server MUST use the same HTTP version.







<span class="grey">Hickman, et al.              Informational                     [Page 21]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-22" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   Test Duration:
      Time, expressed in seconds, for which the virtual client(s) will
      sustain the attempted GET request rate.  It is RECOMMENDED that
      the duration be at least 30 seconds.

   Requests per connection:
      Number of object requests per connection.

   Object Size:
      Defines the number of bytes, excluding any bytes associated with
      the HTTP header, to be transferred in response to an HTTP 1.1 or
      higher GET request.

<span class="h4"><a class="selflink" id="section-5.7.3" href="#section-5.7.3">5.7.3</a> Procedure</span>

   This test will employ an iterative search algorithm to determine the
   maximum transaction rate that the DUT/SUT can sustain.

   For each iteration, HTTP 1.1 or higher virtual client(s) will vary
   the aggregate GET request rate offered to HTTP 1.1 or higher
   server(s).  The virtual client(s) will maintain the offered request
   rate for the defined test duration.

   If the virtual client(s) make multiple HTTP GET requests per
   connection, it MUST request the same object size for each GET
   request.  Multiple tests MAY be performed with different object
   sizes.

<span class="h4"><a class="selflink" id="section-5.7.4" href="#section-5.7.4">5.7.4</a> Measurements</span>

   Maximum Transaction Rate:
      The maximum rate at which all transactions, that is all
      requests/responses cycles, are completed.

   Transaction Time:
      The test instrument SHOULD measure minimum, maximum and average
      transaction times.  The transaction time will start when the
      virtual client issues the GET request and end when the requesting
      virtual client receives the last bit of the requested object.

<span class="h4"><a class="selflink" id="section-5.7.5" href="#section-5.7.5">5.7.5</a> Reporting Format</span>

   The test report MUST conform to the reporting requirements set in
   <a href="#section-4">section 4</a>, Test Setup.







<span class="grey">Hickman, et al.              Informational                     [Page 22]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-23" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


<span class="h5"><a class="selflink" id="section-5.7.5.1" href="#section-5.7.5.1">5.7.5.1</a> Application-Layer reporting</span>

   The test report MUST note the test duration, object size, requests
   per connection, minimum transaction time, maximum transaction time,
   average transaction time and maximum transaction rate measured

   The intermediate results of the search algorithm MAY be reported in a
   table format with a column for each iteration.  There SHOULD be rows
   for the GET request attempt rate, number of requests attempted,
   number and percentage of requests completed, number of responses
   attempted, number and percentage of responses completed, minimum
   transaction time, average transaction time and maximum transaction
   time.

   Version information:
      The test report MUST note the version of HTTP client(s) and
      server(s).

<span class="h5"><a class="selflink" id="section-5.7.5.2" href="#section-5.7.5.2">5.7.5.2</a> Transport-Layer</span>

   The test report MUST note the close method, close direction, number
   of connections established and number of connections torn down.

   The intermediate results of the search algorithm MAY be reported in a
   table format with a column for each iteration.  There SHOULD be rows
   for the number of connections attempted, number and percentage of
   connections completed, number and percentage of connection tear downs
   completed.  The table MAY be combined with the application layer
   reporting, provided the table identify this as transport layer
   measurement.

<span class="h3"><a class="selflink" id="section-5.8" href="#section-5.8">5.8</a>  Illegal Traffic Handling</span>

<span class="h4"><a class="selflink" id="section-5.8.1" href="#section-5.8.1">5.8.1</a> Objective</span>

   To characterize the behavior of the DUT/SUT when presented with a
   combination of both legal and Illegal [<a href="#ref-1" title="&quot;Benchmarking Terminology for Firewall Devices&quot;">1</a>] traffic.  Note that Illegal
   traffic does not refer to an attack, but traffic which has been
   explicitly defined by a rule(s) to drop.

<span class="h4"><a class="selflink" id="section-5.8.2" href="#section-5.8.2">5.8.2</a> Setup Parameters</span>

   Setup parameters will use the same parameters as specified in the
   HTTP transfer rate test (<a href="#section-5.6.2">Section 5.6.2</a>).  In addition, the following
   setup parameters MUST be defined:






<span class="grey">Hickman, et al.              Informational                     [Page 23]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-24" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   Illegal traffic percentage:
      Percentage of HTTP 1.1 or higher connections which have been
      explicitly defined in a rule(s) to drop.

<span class="h4"><a class="selflink" id="section-5.8.3" href="#section-5.8.3">5.8.3</a> Procedure</span>

   Each HTTP 1.1 or higher client will request one or more objects from
   an HTTP 1.1 or higher server using one or more HTTP GET requests over
   each connection.  The aggregate number of connections attempted,
   defined by number of connections, MUST be evenly divided among all of
   the participating virtual clients.

   The virtual client(s) MUST offer the connection requests, both legal
   and illegal, in an evenly distributed manner.  Many firewalls have
   the capability to filter on different traffic criteria (IP addresses,
   Port numbers, etc.).  Multiple iterations of this test MAY be run
   with the DUT/SUT configured to filter on different traffic criteria.

<span class="h4"><a class="selflink" id="section-5.8.4" href="#section-5.8.4">5.8.4</a> Measurements</span>

   The same measurements as defined in HTTP transfer rate test (<a href="#section-5.6.4">Section</a>
   <a href="#section-5.6.4">5.6.4</a>) SHOULD be performed.  Any forwarding rate measurements MUST
   only include bits which are associated with legal traffic.

<span class="h4"><a class="selflink" id="section-5.8.5" href="#section-5.8.5">5.8.5</a> Reporting Format</span>

   Test reporting format SHOULD be the same as specified in the HTTP
   transfer rate test (<a href="#section-5.6.5">Section 5.6.5</a>).

   In addition, the report MUST note the percentage of illegal HTTP
   connections.

   Failure analysis:
      Test report MUST note the number and percentage of illegal
      connections that were allowed by the DUT/SUT.

<span class="h3"><a class="selflink" id="section-5.9" href="#section-5.9">5.9</a> IP Fragmentation Handling</span>

<span class="h4"><a class="selflink" id="section-5.9.1" href="#section-5.9.1">5.9.1</a> Objective</span>

   To determine the performance impact when the DUT/SUT is presented
   with IP fragmented traffic.  IP packets which have been fragmented,
   due to crossing a network that supports a smaller MTU (Maximum
   Transmission Unit) than the actual IP packet, may require the
   firewall to perform re-assembly prior to the rule set being applied.






<span class="grey">Hickman, et al.              Informational                     [Page 24]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-25" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   While IP fragmentation is a common form of attack, either on the
   firewall itself or on internal hosts, this test will focus on
   determining how the additional processing associated with the re-
   assembly of the packets have on the forwarding rate of the DUT/SUT.
   <a href="./rfc1858">RFC 1858</a> addresses some fragmentation attacks that get around IP
   filtering processes used in routers and hosts.

<span class="h4"><a class="selflink" id="section-5.9.2" href="#section-5.9.2">5.9.2</a> Setup Parameters</span>

   The following parameters MUST be defined.

<span class="h5"><a class="selflink" id="section-5.9.2.1" href="#section-5.9.2.1">5.9.2.1</a> Non-Fragmented Traffic Parameters</span>

   Setup parameters will be the same as defined in the HTTP transfer
   rate test (Sections <a href="#section-5.6.2.1">5.6.2.1</a> and <a href="#section-5.6.2.2">5.6.2.2</a>).

<span class="h5"><a class="selflink" id="section-5.9.2.2" href="#section-5.9.2.2">5.9.2.2</a> Fragmented Traffic Parameters</span>

   Packet size:
      Number of bytes in the IP/UDP packet, exclusive of link-layer
      headers and checksums, prior to fragmentation.

   MTU:
      Maximum transmission unit, expressed in bytes.  For testing
      purposes, this MAY be configured to values smaller than the MTU
      supported by the link layer.

   Intended Load:
      Intended load, expressed as percentage of media utilization.

<span class="h4"><a class="selflink" id="section-5.9.3" href="#section-5.9.3">5.9.3</a> Procedure</span>

   Each HTTP 1.1 or higher client will request one or more objects from
   an HTTP 1.1 or higher server using one or more HTTP GET requests over
   each connection.  The aggregate number of connections attempted,
   defined by number of connections, MUST be evenly divided among all of
   the participating virtual clients.  If the virtual client(s) make
   multiple HTTP GET requests per connection, it MUST request the same
   object size for each GET request.

   A test instrument attached to the unprotected side of the network,
   will offer a unidirectional stream of unicast fragmented IP/UDP
   traffic, targeting a server attached to either the protected or DMZ
   segment. The test instrument MUST offer the unidirectional stream
   over the duration of the test, that is, duration over which the HTTP
   traffic is being offered.





<span class="grey">Hickman, et al.              Informational                     [Page 25]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-26" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   Baseline measurements SHOULD be performed with IP filtering deny
   rule(s) to filter fragmented traffic.  If the DUT/SUT has logging
   capability, the log SHOULD be checked to determine if it contains the
   correct information regarding the fragmented traffic.

   The test SHOULD be repeated with the DUT/SUT rule set changed to
   allow the fragmented traffic through.  When running multiple
   iterations of the test, it is RECOMMENDED to vary the MTU while
   keeping all other parameters constant.

   Then setup the DUT/SUT to the policy or rule set the manufacturer
   required to be defined to protect against fragmentation attacks and
   repeat the measurements outlined in the baseline procedures.

<span class="h4"><a class="selflink" id="section-5.9.4" href="#section-5.9.4">5.9.4</a> Measurements</span>

   Test instrument SHOULD perform the same measurements as defined in
   HTTP test (<a href="#section-5.6.4">Section 5.6.4</a>).

   Transmitted UDP/IP Packets:
      Number of UDP packets transmitted by client.

   Received UDP/IP Packets:
      Number of UDP/IP Packets received by server.

<span class="h4"><a class="selflink" id="section-5.9.5" href="#section-5.9.5">5.9.5</a> Reporting Format</span>

<span class="h5"><a class="selflink" id="section-5.9.5.1" href="#section-5.9.5.1">5.9.5.1</a> Non-Fragmented Traffic</span>

   The test report SHOULD be the same as described in <a href="#section-5.6.5">section 5.6.5</a>.
   Note that any forwarding rate measurements for the HTTP traffic
   excludes any bits associated with the fragmented traffic which may be
   forward by the DUT/SUT.

<span class="h5"><a class="selflink" id="section-5.9.5.2" href="#section-5.9.5.2">5.9.5.2</a> Fragmented Traffic</span>

   The test report MUST note the packet size, MTU size, intended load,
   number of UDP/IP packets transmitted and number of UDP/IP packets
   forwarded.  The test report SHOULD also note whether or not the
   DUT/SUT forwarded the offered UDP/IP traffic fragmented.

<span class="h3"><a class="selflink" id="section-5.10" href="#section-5.10">5.10</a> Latency</span>

<span class="h4"><a class="selflink" id="section-5.10.1" href="#section-5.10.1">5.10.1</a> Objective</span>

   To determine the latency of network-layer or application-layer data
   traversing the DUT/SUT. <a href="./rfc1242">RFC 1242</a> [<a href="#ref-3" title="&quot;Benchmarking Terminology for Network Interconnection Devices&quot;">3</a>] defines latency.




<span class="grey">Hickman, et al.              Informational                     [Page 26]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-27" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


<span class="h4"><a class="selflink" id="section-5.10.2" href="#section-5.10.2">5.10.2</a> Setup Parameters</span>

   The following parameters MUST be defined:

<span class="h5"><a class="selflink" id="section-5.10.2.1" href="#section-5.10.2.1">5.10.2.1</a> Network-layer Measurements</span>

   Packet size, expressed as the number of bytes in the IP packet,
   exclusive of link-layer headers and checksums.

   Intended load, expressed as percentage of media utilization.

   Test duration, expressed in seconds.

   The test instruments MUST generate packets with unique timestamp
   signatures.

<span class="h5"><a class="selflink" id="section-5.10.2.2" href="#section-5.10.2.2">5.10.2.2</a> Application-layer Measurements</span>

   Object Size:
      Defines the number of bytes, excluding any bytes associated with
      the HTTP header, to be transferred in response to an HTTP 1.1 or
      higher GET request.  The minimum object size supported by the
      media SHOULD be used, but other object sizes MAY be used as well.

   Connection type:
      The test instrument MUST use one HTTP 1.1 or higher connection for
      latency measurements.

   Number of objects requested.

   Number of objects transferred.

   Test duration, expressed in seconds.

   Test instruments MUST generate packets with unique timestamp
   signatures.

<span class="h4"><a class="selflink" id="section-5.10.3" href="#section-5.10.3">5.10.3</a> Network-layer procedure</span>

   A client will offer a unidirectional stream of unicast packets to a
   server.  The packets MUST use a connectionless protocol like IP or
   UDP/IP.

   The test instrument MUST offer packets in a steady state.  As noted
   in the latency discussion in <a href="./rfc2544">RFC 2544</a> [<a href="#ref-2" title="&quot;Benchmarking Methodology for Network Interconnect Devices&quot;">2</a>], latency measurements MUST
   be taken at the throughput level, that is, at the highest offered
   load with zero packet loss.  Measurements taken at the throughput
   level are the only ones that can legitimately be termed latency.



<span class="grey">Hickman, et al.              Informational                     [Page 27]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-28" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   It is RECOMMENDED that implementers use offered loads not only at the
   throughput level, but also at load levels that are less than or
   greater than the throughput level.  To avoid confusion with existing
   terminology, measurements from such tests MUST be labeled as delay
   rather than latency.

   It is RECOMMENDED to perform the latency measurements with different
   packet sizes.  When testing with different packet sizes the DUT/SUT
   configuration MUST remain the same.

   If desired, a step test MAY be used in which offered loads increment
   or decrement through a range of load levels.

   The duration of the test portion of each trial MUST be at least 30
   seconds.

<span class="h4"><a class="selflink" id="section-5.10.4" href="#section-5.10.4">5.10.4</a> Application layer procedure</span>

   An HTTP 1.1 or higher client will request one or more objects from an
   HTTP 1.1 or higher server using one or more HTTP GET requests.  If
   the test instrument makes multiple HTTP GET requests, it MUST request
   the same-sized object each time.  Multiple iterations of this test
   may be performed with objects of different sizes.

   Implementers MAY configure the test instrument to run for a fixed
   duration.  In this case, the test instrument MUST report the number
   of objects requested and returned for the duration of the test.  For
   fixed-duration tests it is RECOMMENDED that the duration be at least
   30 seconds.

<span class="h4"><a class="selflink" id="section-5.10.5" href="#section-5.10.5">5.10.5</a> Measurements</span>

   Minimum delay:
      The smallest delay incurred by data traversing the DUT/SUT at the
      network layer or application layer, as appropriate.

   Maximum delay:
      The largest delay incurred by data traversing the DUT/SUT at the
      network layer or application layer, as appropriate.

   Average delay:
      The mean of all measurements of delay incurred by data traversing
      the DUT/SUT at the network layer or application layer, as
      appropriate.







<span class="grey">Hickman, et al.              Informational                     [Page 28]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-29" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   Delay distribution:
      A set of histograms of all delay measurements observed for data
      traversing the DUT/SUT at the network layer or application layer,
      as appropriate.

<span class="h4"><a class="selflink" id="section-5.10.6" href="#section-5.10.6">5.10.6</a> Network-layer reporting format</span>

   The test report MUST note the packet size(s), offered load(s) and
   test duration used. In addition, the test report MUST conform to the
   reporting requirements set in <a href="#section-4">section 4</a>, Test Setup.

   The latency results SHOULD be reported in the format of a table with
   a row for each of the tested packet sizes.  There SHOULD be columns
   for the packet size, the intended rate, the offered rate, and the
   resultant latency or delay values for each test.

<span class="h4"><a class="selflink" id="section-5.10.7" href="#section-5.10.7">5.10.7</a> Application-layer reporting format</span>

   The test report MUST note the object size(s) and number of requests
   and responses completed.  If applicable, the report MUST note the
   test duration if a fixed duration was used. In addition, the test
   report MUST conform to the reporting requirements set in <a href="#section-4">section 4</a>,
   Test Setup.

   The latency results SHOULD be reported in the format of a table with
   a row for each of the object sizes.  There SHOULD be columns for the
   object size, the number of completed requests, the number of
   completed responses, and the resultant latency or delay values for
   each test.

   Failure analysis:
      The test report SHOULD indicate the number and percentage of HTTP
      GET request or responses that failed to complete within the test
      duration.

   Version information:
      The test report MUST note the version of HTTP client and server.

<span class="h2"><a class="selflink" id="section-6" href="#section-6">6</a>. References</span>

<span class="h3"><a class="selflink" id="section-6.1" href="#section-6.1">6.1</a>  Normative References</span>

   [<a id="ref-1">1</a>]  Newman, D., "Benchmarking Terminology for Firewall Devices", <a href="./rfc2647">RFC</a>
        <a href="./rfc2647">2647</a>, August 1999.

   [<a id="ref-2">2</a>]  Bradner, S. and J. McQuaid, "Benchmarking Methodology for
        Network Interconnect Devices", <a href="./rfc2544">RFC 2544</a>, March 1999.




<span class="grey">Hickman, et al.              Informational                     [Page 29]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-30" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   [<a id="ref-3">3</a>]  Bradner, S., "Benchmarking Terminology for Network
        Interconnection Devices", <a href="./rfc1242">RFC 1242</a>, July 1991.

   [<a id="ref-4">4</a>]  Mandeville, R., "Benchmarking Terminology for LAN Switching
        Devices", <a href="./rfc2285">RFC 2285</a>, February 1998.

   [<a id="ref-5">5</a>]  Mandeville, R. and J. Perser, "Benchmarking Methodology for LAN
        Switching Devices", <a href="./rfc2889">RFC 2889</a>, August 2000.

<span class="h3"><a class="selflink" id="section-6.2" href="#section-6.2">6.2</a>  Informative References</span>

   [<a id="ref-6">6</a>]  Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L.,
        Leach, P. and T. Berners-Lee, "Hypertext Transfer Protocol -
        HTTP/1.1", <a href="./rfc2616">RFC 2616</a>, June 1999.

   [<a id="ref-7">7</a>]  Clark, D., "IP Datagram Reassembly Algorithm", <a href="./rfc815">RFC 815</a>, July
        1982.

   [<a id="ref-8">8</a>]  Postel, J., "Transmission Control Protocol", STD 7, <a href="./rfc793">RFC 793</a>,
        September 1981.

<span class="h2"><a class="selflink" id="section-7" href="#section-7">7</a>. Security Considerations</span>

   The primary goal of this document is to provide methodologies in
   benchmarking firewall performance. While there is some overlap
   between performance and security issues, assessment of firewall
   security is outside the scope of this document.
























<span class="grey">Hickman, et al.              Informational                     [Page 30]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-31" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


APPENDIX A: HTTP (HyperText Transfer Protocol)

   The most common versions of HTTP in use today are HTTP/1.0 and
   HTTP/1.1 with the main difference being in regard to persistent
   connections.  HTTP 1.0, by default, does not support persistent
   connections.  A separate TCP connection is opened up for each GET
   request the client wants to initiate and closed after the requested
   object transfer is completed.  While some implementations HTTP/1.0
   supports persistence through the use of a keep-alive, there is no
   official specification for how the keep-alive operates. In addition,
   HTTP 1.0 proxies do support persistent connection as they do not
   recognize the connection header.

   HTTP/1.1, by default, does support persistent connection and is
   therefore the version that is referenced in this methodology. Proxy
   based DUT/SUTs may monitor the TCP connection and after a timeout,
   close the connection if no activity is detected.  The duration of
   this timeout is not defined in the HTTP/1.1 specification and will
   vary between DUT/SUTs.  If the DUT/SUT closes inactive connections,
   the aging timer on the DUT SHOULD be configured for a duration that
   exceeds the test time.

   While this document cannot foresee future changes to HTTP and it
   impact on the methodologies defined herein, such changes should be
   accommodated for so that newer versions of HTTP may be used in
   benchmarking firewall performance.

APPENDIX B: Connection Establishment Time Measurements

   Some connection oriented protocols, such as TCP, involve an odd
   number of messages when establishing a connection.  In the case of
   proxy based DUT/SUTs, the DUT/SUT will terminate the connection,
   setting up a separate connection to the server.  Since, in such
   cases, the test instrument does not own both sides of the connection,
   measurements will be made two different ways.  While the following
   describes the measurements with reference to TCP, the methodology may
   be used with other connection oriented protocols which involve an odd
   number of messages.

   When testing non-proxy based DUT/SUTs , the establishment time shall
   be directly measured and is considered to be from the time the first
   bit of the first SYN packet is transmitted by the client to the time
   the last bit of the final ACK in the three-way handshake is received
   by the target server.







<span class="grey">Hickman, et al.              Informational                     [Page 31]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-32" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


   If the DUT/SUT is proxy based, the connection establishment time is
   considered to be from the time the first bit of the first SYN packet
   is transmitted by the client to the time the client transmits the
   first bit of the first acknowledged TCP datagram (t4-t0 in the
   following timeline).

      t0: Client sends a SYN.
      t1: Proxy sends a SYN/ACK.
      t2: Client sends the final ACK.
      t3: Proxy establishes separate connection with server.
      t4: Client sends TCP datagram to server.
      *t5: Proxy sends ACK of the datagram to client.

   * While t5 is not considered part of the TCP connection
   establishment, acknowledgement of t4 must be received for the
   connection to be considered successful.

APPENDIX C: Connection Tear Down Time Measurements

   While TCP connections are full duplex, tearing down of such
   connections are performed in a simplex fashion, that is, FIN segments
   are sent by each host/device terminating each side of the TCP
   connection.

   When making connection tear down times measurements, such
   measurements will be made from the perspective of the entity, that
   is, virtual client/server initiating the connection tear down
   request.  In addition, the measurement will be performed in the same
   manner, independent of whether or not the DUT/SUT is proxy-based. The
   connection tear down will be considered the interval between the
   transmission of the first bit of the first TCP FIN packet transmitted
   by the virtual client or server, whichever is applicable, requesting
   a connection tear down to receipt of the last bit of the
   corresponding ACK packet on the same virtual client/server interface.

















<span class="grey">Hickman, et al.              Informational                     [Page 32]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-33" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


Authors' Addresses

   Brooks Hickman
   Spirent Communications
   26750 Agoura Road
   Calabasas, CA 91302
   USA

   Phone: + 1 818 676 2412
   EMail: brooks.hickman@spirentcom.com


   David Newman
   Network Test Inc.
   31324 Via Colinas, Suite 113
   Westlake Village, CA 91362-6761
   USA

   Phone: + 1 818 889-0011
   EMail: dnewman@networktest.com


   Saldju Tadjudin
   Spirent Communications
   26750 Agoura Road
   Calabasas, CA 91302
   USA

   Phone: + 1 818 676 2468
   EMail: Saldju.Tadjudin@spirentcom.com


   Terry Martin
   GVNW Consulting Inc.
   8050 SW Warm Springs Road
   Tualatin Or. 97062
   USA

   Phone: + 1 503 612 4422
   EMail: tmartin@gvnw.com











<span class="grey">Hickman, et al.              Informational                     [Page 33]</span></pre>
<hr class='noprint'/><!--NewPage--><pre class='newpage'><span id="page-34" ></span>
<span class="grey"><a href="./rfc3511">RFC 3511</a>          Methodology for Firewall Performance        April 2003</span>


Full Copyright Statement

   Copyright (C) The Internet Society (2003).  All Rights Reserved.

   This document and translations of it may be copied and furnished to
   others, and derivative works that comment on or otherwise explain it
   or assist in its implementation may be prepared, copied, published
   and distributed, in whole or in part, without restriction of any
   kind, provided that the above copyright notice and this paragraph are
   included on all such copies and derivative works.  However, this
   document itself may not be modified in any way, such as by removing
   the copyright notice or references to the Internet Society or other
   Internet organizations, except as needed for the purpose of
   developing Internet standards in which case the procedures for
   copyrights defined in the Internet Standards process must be
   followed, or as required to translate it into languages other than
   English.

   The limited permissions granted above are perpetual and will not be
   revoked by the Internet Society or its successors or assigns.

   This document and the information contained herein is provided on an
   "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING
   TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING
   BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION
   HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF
   MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Acknowledgement

   Funding for the RFC Editor function is currently provided by the
   Internet Society.



















Hickman, et al.              Informational                     [Page 34]
</pre>