<?xml version="1.0"?>
<!DOCTYPE rfc>
<?rfc private="remctl Protocol" toc="yes" symrefs="yes"?>
<rfc category="info" docName="draft-allbery-remctl-00" ipr="trust200902"
     submissionType="independent">
  <front>
    <title abbrev="remctl">remctl: Remote Authenticated Command Service</title>
    <author initials='R.' surname='Allbery' fullname='Russ Allbery'>
      <organization>AURA</organization>
      <address>
        <email>eagle@eyrie.org</email>
        <uri>https://www.eyrie.org/~eagle/</uri>
      </address>
    </author>
    <date month='January' year='2014' />

    <abstract>
      <t>This document specifies the remctl wire protocol, used to send
      commands and arguments to a remote system and receive the results of
      executing that command.  The protocol uses GSS-API and Kerberos v5
      for authentication, confidentiality, and integrity protection.  Both
      the current (version 3) protocol and the older version 1 protocol
      are described.  The version 1 protocol should only be implemented
      for backward compatibility.</t>
    </abstract>
  </front>

  <middle>
    <section anchor='format' title='Basic Packet Format'>
      <t>The remctl network protocol consists of data packets sent from a
      client to a server or a server to a client over a TCP connection.
      The remctl protocol may be used over any port, but the
      IANA-registered port and the RECOMMENDED default for the protocol is
      4373.  Each data packet has the following format:</t>

      <figure>
        <artwork>
    1 octet     flags
    4 octets    length
    &lt;length>    data payload
        </artwork>
      </figure>

      <t>The total size of each token, including the five octet prefix,
      MUST NOT be larger than 1,048,576 octets (1MB).</t>

      <figure>
        <preamble>The flag octet contains one or more of the following
        values, combined with binary xor:</preamble>

        <artwork>
    0x01        TOKEN_NOOP
    0x02        TOKEN_CONTEXT
    0x04        TOKEN_DATA
    0x08        TOKEN_MIC
    0x10        TOKEN_CONTEXT_NEXT
    0x20        TOKEN_SEND_MIC
    0x40        TOKEN_PROTOCOL
        </artwork>

        <postamble>Only TOKEN_CONTEXT, TOKEN_CONTEXT_NEXT, TOKEN_DATA, and
        TOKEN_PROTOCOL are used for packets for versions 2 and 3 of the
        protocol.  The other flags are used only with the legacy version 1
        protocol.</postamble>
      </figure>

      <t>The length field is a four-octet length in network byte order,
      specifying the number of octets in the following data payload.</t>

      <t>The data payload is empty, the results of gss_accept_sec_context,
      the results of gss_init_sec_context, or a data payload protected
      with gss_wrap.  The length of the data passed to gss_wrap MUST NOT
      be larger than 65,536 octets (64KB), even if the underlying Kerberos
      implementation supports longer input buffers.</t>
    </section>

    <section anchor='proto3' title='Network Protocol (version 3)'>
      <section anchor='packet' title='Session Sequence'>
        <t>A remctl connection is always initiated by a client opening a
        TCP connection to a server.  The protocol then proceeds as
        follows:
          <list style='numbers'>
            <t>Client sends message with an empty payload and flags
            TOKEN_NOOP, TOKEN_CONTEXT_NEXT, and TOKEN_PROTOCOL (0x51).  If
            the client doesn't include TOKEN_PROTOCOL, it is speaking the
            version 1 protocol, and the server MUST either drop the
            connection or fall back to the version 1 protocol.  This
            initial message is useless in a pure version 2 or 3 protocol
            world and is done only for backward compatibility with the
            version 1 protocol.</t>

            <t>Client calls gss_init_sec_context and sends the results as
            the message body with flags TOKEN_CONTEXT and TOKEN_PROTOCOL
            (0x42).  The client MUST pass GSS_C_MUTUAL_FLAG,
            GSS_C_CONF_FLAG, and GSS_C_INTEG_FLAG as requested flags to
            gss_init_sec_context and SHOULD pass GSS_C_REPLAY_FLAG and
            GSS_C_SEQUENCE_FLAG.</t>

            <t>Server replies with the results of gss_accept_sec_context
            and flags TOKEN_CONTEXT and TOKEN_PROTOCOL (0x42).  If the
            server doesn't include TOKEN_PROTOCOL in the flags, it is
            speaking the version 1 protocol, and the client MUST either
            drop the connection or fall back to the version 1
            protocol.</t>

            <t>Client passes data to gss_init_sec_context and replies with
            the results and TOKEN_CONTEXT and TOKEN_PROTOCOL (0x42).  The
            client must pass GSS_C_MUTUAL_FLAG, GSS_C_CONF_FLAG, and
            GSS_C_INTEG_FLAG as requested flags and SHOULD pass
            GSS_C_REPLAY_FLAG and GSS_C_SEQUENCE_FLAG.</t>

            <t>Server and client repeat, passing in the payload from the
            last packet from the other side, for as long as GSS-API
            indicates that continuation is required.  If either side drops
            TOKEN_PROTOCOL from the flags, it is an considered an error
            and the connect MUST be dropped.  (This could be a
            down-negotiation attack.)  After the establishment of the
            security context, both client and server MUST confirm that
            GSS_C_MUTUAL_FLAG, GSS_C_CONF_FLAG, and GSS_C_INTEG_FLAG are
            set in the resulting security context and MUST immediately
            close the connection if this is not the case.</t>

            <t>After the security context has been established, the client
            and server exchange commands and responses as described below.
            All commands are sent with flags TOKEN_DATA and TOKEN_PROTOCOL
            (0x44) and the data payload of all packets is protected with
            gss_wrap.  The conf_req_flag parameter of gss_wrap MUST be set
            to non-zero, requesting both confidentiality and integrity
            services.</t>
          </list>
        </t>
      </section>

      <section anchor='messages' title='Message Format'>
        <t>All client and server messages will use the following format
        inside the data payload.  This is the format of the message before
        passing it to gss_wrap for confidentiality and integrity
        protection.</t>

        <figure>
          <artwork>
    1 octet     protocol version
    1 octet     message type
    &lt;command-specific data>
          </artwork>
        </figure>

        <t>The protocol version sent for all messages should be 2 with the
        exception of MESSAGE_NOOP, which should have a protocol version of
        3.  The version 1 protocol does not use this message format, and
        therefore a protocol version of 1 is invalid.  See below for
        protocol version negotiation.</t>

        <figure>
          <preamble>The message type is one of the following
          constants:</preamble>

          <artwork>
    1   MESSAGE_COMMAND
    2   MESSAGE_QUIT
    3   MESSAGE_OUTPUT
    4   MESSAGE_STATUS
    5   MESSAGE_ERROR
    6   MESSAGE_VERSION
    7   MESSAGE_NOOP
          </artwork>
        </figure>

        <t>The first two message types are client messages and MUST NOT be
        sent by the server.  The remaining message types except for
        MESSAGE_NOOP are server messages and MUST NOT by sent by the
        client.</t>

        <t>All of these message types were introduced in protocol version
        2 except for MESSAGE_NOOP, which is a protocol version 3
        message.</t>
      </section>

      <section anchor='negotiation' title='Protocol Version Negotiation'>
        <t>If the server ever receives a message from a client that claims a
        protocol version higher than the server supports, the server MUST
        otherwise ignore the contents of the message and SHOULD respond with
        a message type of MESSAGE_VERSION and the following message
        payload:</t>

        <figure>
          <artwork>
    1 octet     highest supported version
          </artwork>
        </figure>

        <t>The client MUST then either send only messages supported at
        that protocol version or lower or send MESSAGE_QUIT and close the
        connection.</t>

        <t>Currently, there are only two meaningful values for the highest
        supported version: 3, which indicates everything in this
        specification is supported, or 2, which indicates that everything
        except MESSAGE_NOOP is supported.</t>
      </section>

      <section anchor='command' title='MESSAGE_COMMAND'>
        <t>Most client messages will be of type MESSAGE_COMMAND, which has
        the following format:</t>

        <figure>
          <artwork>
    1 octet     keep-alive flag
    1 octet     continue status
    4 octets    number of arguments
    4 octets    argument length
    &lt;length>    argument
    ...
          </artwork>
        </figure>

        <t>If the keep-alive flag is 0, the server SHOULD close the
        connection after processing the command.  If it is 1, the server
        SHOULD leave the connection open (up to a timeout period) and wait
        for more commands.  This is similar to HTTP keep-alive.</t>

        <t>If the continue status is 0, it indicates that this is the
        complete command.  If the continue status is 1, it indicates that
        there is more data coming.  The server should accept the data
        sent, buffer it, and wait for additional messages before running
        the command or otherwise responding.  If the the continue status
        is 2, it indicates that this message is logically a part of the
        previous message (which MUST have had a continue status of 1 or 2)
        and still has more data coming.  If the continue status is 3, it
        says that this message is logically part of the previous message,
        like 2, but it also says that this is the end of the command.</t>

        <t>A continuation of a message starts with the keep-alive flag and
        continue status and then the next chunk of data.  To reconstruct a
        continued message, remove the first two octets from each chunk and
        concatenate the pieces together.  The result is the portion of a
        MESSAGE_COMMAND starting with the number of arguments.</t>

        <t>The current implementation permits messages to be broken into
        multiple MESSAGE_COMMANDs even in the middle of the number of
        arguments or an argument length.  In other words, the first three
        octets of the number of arguments could be in the first
        MESSAGE_COMMAND (with continue status 1) and the last octet would
        then be in the next MESSAGE_COMMAND (with continue status 2 or 3).
        However, the client SHOULD NOT take advantage of this support and
        SHOULD NOT split an argument count or argument length across
        multiple messages, since this support may be dropped in a
        subsequent protocol revision.</t>

        <t>For as long as the continue status is 1 or 2, the next message
        from the client MUST be either another MESSAGE_COMMAND with a
        continue status of 2 or 3 or a MESSAGE_QUIT.  In other words,
        other message types MUST NOT be intermixed with continued
        commands, but MESSAGE_QUIT may be sent by the client in the middle
        of a continued command to abort that command.  If the server
        receives MESSAGE_QUIT from the client before receiving a
        MESSAGE_COMMAND with a status of 3 (indicating the end of the
        command), the command MUST be discarded and not executed.</t>

        <t>If a client sends an invalid sequence of MESSAGE_COMMAND
        messages that violate the continuation rules described above, the
        server SHOULD reply with a MESSAGE_ERROR message, generally with
        one of the ERROR_BAD_TOKEN, ERROR_UNKNOWN_MESSAGE,
        ERROR_BAD_COMMAND, or ERROR_UNEXPECTED_MESSAGE error codes.  It
        MUST discard the partial command without acting on it.  The client
        cannot correct an error in a continued MESSAGE_COMMAND stream by
        resending the previous part.  It MUST start again at the beginning
        with a MESSAGE_COMMAND with a continue status of 0 or 1.</t>

        <t>Number of arguments is a four-octet number in network byte
        order that gives the total number of command arguments.  For each
        argument, there is then a length and argument data pair, where the
        length is a four-octet number in network byte order indicating the
        number of octets of data in the following argument.  Argument
        length may be 0.  Commands with no arguments are permitted by the
        protocol.</t>

        <t>Servers may impose limits on the number of arguments and the
        size of argument data to limit resource usage.  If the client
        message exceeds one of those limits, the server MUST respond with
        MESSAGE_ERROR with an error code of ERROR_TOOMANY_ARGS or
        ERROR_TOOMUCH_DATA as appropriate.</t>
      </section>

      <section anchor='output' title='MESSAGE_OUTPUT and MESSAGE_STATUS'>
        <t>The server response to MESSAGE_COMMAND is zero or more
        MESSAGE_OUTPUT messages followed by either a MESSAGE_STATUS or a
        MESSAGE_ERROR response.  Each MESSAGE_OUTPUT message has the
        following format:</t>

        <figure>
          <artwork>
    1 octet     output stream
    4 octets    output length
    &lt;length>    output
          </artwork>
        </figure>

        <t>The output stream is either 1 for standard output or 2 for
        standard error.  Output length is a four-octet number in network
        byte order that specifies the length of the following output
        data.</t>

        <t>The MESSAGE_STATUS message has the following format:</t>

        <figure>
          <artwork>
    1 octet     exit status
          </artwork>
        </figure>

        <t>MESSAGE_STATUS indicates the command has finished and returns
        the final exit stauts of the command.  Exit status is 0 for
        success and non-zero for failure, where the meaning of non-zero
        exit statuses is left to the application to define.  (This is
        identical to a Unix command exit status.)</t>

        <t>Unless the MESSAGE_COMMAND message from the client had the
        keep-alive flag set to 1, the server MUST close the network
        connection immediately after sending the MESSAGE_STATUS response
        message.</t>
      </section>

      <section anchor='error' title='MESSAGE_ERROR'>
        <t>At any point before sending MESSAGE_STATUS, the server may
        respond with MESSAGE_ERROR if some error occurred.  This can be
        the first response after a MESSAGE_COMMAND, or it may be sent
        after one or more MESSAGE_OUTPUT messages.  The format of
        MESSAGE_ERROR is as follows:</t>

        <figure>
          <artwork>
    4 octets    error code
    4 octets    message length
    &lt;length>    error message
          </artwork>
        </figure>

        <t>The error code is a four-octet number in network byte order
        indicating the type of error.  The error code may be one of the
        following values:</t>

        <figure>
          <artwork>
    1  ERROR_INTERNAL           Internal server failure
    2  ERROR_BAD_TOKEN          Invalid format in token
    3  ERROR_UNKNOWN_MESSAGE    Unknown message type
    4  ERROR_BAD_COMMAND        Invalid command format in token
    5  ERROR_UNKNOWN_COMMAND    Unknown command
    6  ERROR_ACCESS             Access denied
    7  ERROR_TOOMANY_ARGS       Argument count exceeds server limit
    8  ERROR_TOOMUCH_DATA       Argument size exceeds server limit
    9  ERROR_UNEXPECTED_MESSAGE Message type not valid now
          </artwork>
        </figure>

        <t>Additional error codes may be added without changing the
        version of the remctl protocol, so clients MUST accept error codes
        other than the ones above.</t>

        <t>The message length is a four-octet number in network byte order
        that specifies the length in octets of the following error
        message.  The error message is a free-form informational message
        intended for human consumption and MUST NOT be interpreted by an
        automated process.  Software should instead use the error
        code.</t>

        <t>Unless the MESSAGE_COMMAND message from the client had the
        keep-alive flag set to 1, the server MUST close the network
        connection immediately after sending the MESSAGE_ERROR response
        message.  Otherwise, the server SHOULD still honor that flag,
        although the server MAY terminate the connection after an
        unreasonable number of errors.</t>
      </section>

      <section anchor='quit' title='MESSAGE_QUIT'>
        <t>MESSAGE_QUIT is a way of terminating the connection cleanly if
        the client asked for keep-alive and then decided not to use it.
        There is no message body.  Upon receiving this message, the server
        MUST immediately close the connection.</t>
      </section>

      <section anchor='noop' title='MESSAGE_NOOP'>
        <t>MESSAGE_NOOP provides a way for a client to keep the connection
        open to a remctl server, including through firewall session
        timeouts and similar network constraints that require periodic
        activity, without sending new commands.  There is no body.  When
        the client sends a MESSAGE_NOOP message, the server replies with a
        MESSAGE_NOOP message.</t>

        <t>Note that MESSAGE_NOOP was introduced in protocol version 3 and
        therefore should be marked accordingly.  Clients should be
        prepared for older servers to reply with MESSAGE_VERSION instead
        of MESSAGE_NOOP.</t>
      </section>
    </section>

    <section anchor='proto1' title='Network Protocol (version 1)'>
      <t>The old network protocol supported only 64KB of data payload,
      only a single command and response, and had some additional
      unnecessary protocol components.  It SHOULD NOT be used by clients,
      but MAY be supported by servers for backward compatibility.  It is
      recognized by the server and client by the lack of TOKEN_PROTOCOL in
      the flags of the initial security context negotiation.</t>

      <t>The old protocol always uses the following steps:
        <list style='numbers'>
          <t>Client opens TCP connection to server.</t>

          <t>Client sends message with flags TOKEN_NOOP and
          TOKEN_CONTEXT_NEXT and an empty payload.</t>

          <t>Client calls gss_init_sec_context and sends message with the
          results and flags TOKEN_CONTEXT.  The client MUST pass
          GSS_C_MUTUAL_FLAG, GSS_C_CONF_FLAG, and GSS_C_INTEG_FLAG as
          requested flags and SHOULD pass GSS_C_REPLAY_FLAG and
          GSS_C_SEQUENCE_FLAG, although the version one protocol does not
          check the results of this negotiation.</t>

          <t>Server replies with the results of gss_accept_sec_context and
          flags TOKEN_CONTEXT.</t>

          <t>Client calls gss_init_sec_context again with the data from
          the server and replies with the results and flags TOKEN_CONTEXT,
          using the same requested flags as described above.</t>

          <t>Server and client repeat, passing in the payload from the
          last packet from the other side, for as long as GSS-API
          indicates that continuation is required.  Each of these packets
          have only TOKEN_CONTEXT set in the flags.</t>

          <t>Client sends command with flags TOKEN_DATA and TOKEN_SEND_MIC
          and the following payload format:  four-octet number of
          arguments, and then for each argument, a four-octet length and
          then the argument value.  All numbers are in network type order.
          The payload MUST be protected with gss_wrap and the
          conf_req_flag parameter of gss_wrap MUST be set to non-zero,
          requesting both confidentiality and integrity services.</t>

          <t>Server accepts and decrypts data, generates a MIC with
          gss_get_mic, and sends the MIC back to the client with flags
          TOKEN_MIC.  This is the only packet that isn't encrypted with
          gss_wrap.  Client receives and then SHOULD verify this MIC.</t>

          <t>Server runs the command, collects the output, and sends the
          output back with flags TOKEN_DATA and the following payload
          format:  four-octet exit status, four-octet data length, data.
          All numbers are in network byte order.  The exit status is 0 if
          there were no errors and non-zero otherwise, where the meaning
          of non-zero values are defined by the application.  The payload
          MUST be protected with gss_wrap with a conf_req_flag set to
          non-zero.</t>

          <t>Server and client close connection.</t>
        </list>
      </t>
    </section>

    <section anchor='security' title='Security Considerations'>
      <t>It would be preferrable to insist on replay and sequence
      protection (GSS_C_REPLAY_FLAG and GSS_C_SEQUENCE_FLAG) for all
      contexts, but some older Kerberos GSS-API implementations don't
      support this and hence it is not mandatory in the protocol.  Clients
      SHOULD always request replay and sequence protection, however, and
      servers MAY require such protection be negotiated.</t>

      <t>The old protocol doesn't provide integrity protection for the
      flags, but since it always follows the same fixed sequence of
      operations, this should pose no security concerns in practice.  The
      new protocol only uses the flag field outside of the encrypted
      section of the packet for initial negotiation and closes the
      connection if the flags aren't what was expected (avoiding a
      down-negotiation attack).</t>

      <t>In the old protocol, the server calculated and sent a MIC back to
      the client, which then verified that the command as received by the
      server was correct.  Not only does GSS-API already provide integrity
      protection, but this verification also happens after the server has
      already started running the command.  It has been dropped in the new
      protocol.</t>

      <t>The old protocol doesn't require the client and server check the
      results of the GSS-API flag negotiation, although all old protocol
      clients passed GSS_C_MUTUAL_FLAG.  However, the old protocol
      requires gss_wrap be used for all payload with conf_req_flag set to
      non-zero, so any context that didn't negotiate confidentiality and
      integrity services would fail later.</t>
    </section>
  </middle>
  <back>
    <section anchor='credits' title='Acknowledgements'>
      <t>The original remctl protocol design was done by Anton Ushakov,
      with input from Russ Allbery and Roland Schemers.  Thank you to
      David Hoffman and Mike Newton for their review of the version 2
      remctl protocol.</t>
    </section>

    <section anchor='license' title='Additional License'>
      <t>This section supplements the Copyright Notice section at the
      start of this document.  It states an additional copyright notice
      and grants a much less restrictive license than the default IETF
      Trust license.  You may copy and distribute this document, with or
      without modification, under your choice of the license specified in
      the Copyright Notice section or the license below.</t>

      <t>Copyright 2006-2009, 2011, 2013-2014 The Board of Trustees of the
      Leland Stanford Junior University</t>

      <t>Copying and distribution of this file, with or without
      modification, are permitted in any medium without royalty provided
      the copyright notice and this notice are preserved.  This file is
      offered as-is, without any warranty.</t>

      <!-- SPDX-License-Identifier: FSFAP -->
    </section>
  </back>
</rfc>