/**
 * @page rpc_system OLA RPC System
 *
 * [TOC]
 *
 * @section rpc_Overview Overview
 * A common question that comes up is "How can I communicate with olad using
 * something other than C++, Python or Java?".
 *
 * The bad news is that unless someone has written a client library for your
 * language, this requires a bit of work. The good news is that there are
 * people willing to help and if you do a good job when writing the client
 * library, we'll incorporate it into the OLA codebase and maintain it going
 * forward.

 * @section rpc_Background Background
 * This section provides an introduction to Remote Procedure Calls (RPC).
 * If you have used RPCs in other situations (JMI, CORBA, Thrift, etc.) you may
 * want to skip this bit.
 *
 * <a href="http://en.wikipedia.org/wiki/Remote_procedure_call">Remote
 * procedure call</a> has some good background reading.
 *
 * On the client side there is a stub, usually a class. Each stub method takes
 * some request data and some sort of callback function that is run when the
 * remote call completes. The stub class is auto-generated from the protocol
 * specification. The stub class is responsible for serializing the request
 * data, sending it to the remote end, waiting for the response, deserializing
 * the response and finally invoking the callback function.
 *
 * On the server side there is a usually a 'Service' class that has to be
 * provided. Usually an abstract base class is generated from the protocol
 * specification and the programmer subclasses this and provides an
 * implementation for each method that can be called.
 *
 * Because the client stub is autogenerated their can be a lot of duplicated
 * code which constructs the request message, sets up the completion callbacks
 * and invokes the method. Rather than forcing the users of the client library
 * to do duplicate all this work, there is usually a layer in between that
 * provides a friendly API and calls into the stub. This layer can be thin
 * (all it does is wrap the stub functions) or thick (a single API call may
 * invoke multiple RPCs).
 *
 * These client APIs can be blocking or asynchronous. Blocking clients force the
 * caller to wait for the RPC to complete before the function call returns.
 * Asynchronous APIs take a callback, in much the same way as the stub itself
 * does.
 *
 * @section rpc_Proto Use of Protocol Buffers
 *
 * OLA uses <a href="https://code.google.com/p/protobuf/">Protocol Buffers</a>
 * as the data serialization format. Protocol buffers allow us to wrap the
 * request and response data up into a message (similar to a C struct) and then
 * serialize the message to binary data in a platform independent way. It's
 * worth reading the
 * <a href="https://developers.google.com/protocol-buffers/docs/overview">
 * Protocol Buffers documentation</a> before going further. You don't need to
 * understand the binary format, but you should be familiar with the Protocol
 * Buffer definition format.
 *
 * As well as data serialization, protobufs also provide a basic framework for
 * building RPC services. Early versions of protobufs came with a <a
 * href="https://developers.google.com/protocol-buffers/docs/reference/cpp-generated#service">
 * service generator</a>. The generic service generator has been deprecated
 * since the 2.4.0 release since code generated was rather inflexible (in
 * trying to be all things to all people it left many needs un-addressed).
 *
 * Let's look at a simple example for a dictionary service, Dictionary.proto.
 * The client sends the word its looking for and the server replies with the
 * definition and maybe some extra information like the pronunciation and a
 * list of similar words.
 * ~~~~~~~~~~~~~~~~~~~~~
   message WordQuery {
     required string word = 1;
   }

   message WordDefinition {
     required string definition = 1;
     optional string pronunciation = 2;
     repeated string similar_words = 3;
   }

   service DictionaryLookup {
    rpc Lookup(WordQuery) returns(WordDefinition);
   }
  ~~~~~~~~~~~~~~~~~~~~~
 *
 * We can generate the C++ code by running:
 *
 *     protoc --plugin=protoc-gen-cppservice=protoc/ola_protoc_plugin \
 *     --cppservice_out ./ Dictionary.proto
 *
 * The generated C++ stub class definition in Dictionary.pb.h looks something
 * like:
  ~~~~~~~~~~~~~~~~~~~~~
  class DictionaryLookup_Stub {
   public:
    DictionaryLookup_Stub(::google::protobuf::RpcChannel* channel);

    void Lookup(\::google::protobuf::RpcController* controller,
                const \::WordQuery* request,
                \::WordDefinition* response,
                \::google::protobuf::Closure* done);
  };
  ~~~~~~~~~~~~~~~~~~~~~
 *
 * As you can see the 2nd and 3rd arguments are the request and response
 * messages (WordQuery and WordDefinition respectively). The 4th argument is
 * the completion callback and the 1st argument keeps track of the outstanding
 * RPC.
 *
 * An implementation of DictionaryLookup_Stub::Lookup(...) is generated in
 * Dictionary.pb.cc. Since the only thing that differs between methods is the
 * request / response types and the method name, the implementation just calls
 * through to the RpcChannel class:
 *
  ~~~~~~~~~~~~~~~~~~~~~
    void DictionaryLookup_Stub::Lookup(\::google::protobuf::RpcController* controller,
                                       const \::WordQuery* request,
                                       \::WordDefinition* response,
                                       \::google::protobuf::Closure* done) {
      channel_->CallMethod(descriptor()->method(0),
                           controller, request, response, done);
    }
  ~~~~~~~~~~~~~~~~~~~~~
 *
 * Protocol buffers doesn't provide an implementation of RpcChannel, since it's
 * implementation specific. Most of the time the RpcChannel uses a TCP
 * connection but you can imagine other data passing mechanisms.
 *
 * As part of writing a new client you'll need to write an implementation of
 * an RpcChannel. The C++ implementation is ola::rpc::RpcChannel.
 *
 * Putting it all together produces the flow shown in the diagram below. The
 * yellow code is auto-generated, the blue code has to be written.
 *
 * \image html rpc.png "The control flow from the Client to the Server"
 *
 *
 * @section rpc_OLA_RPC OLA's Model
 *
 * By default, olad listens on localhost:9010 . Clients open a TCP connection
 * to this ip:port and write serialized protobufs (see \ref rpc_RPCLayer) to
 * the connection.
 *
 * Why TCP and not UDP? Mostly because the original developer was lazy and
 * didn't want to deal with chunking data into packets, when TCP handles this
 * for us. However TCP has it's own problems, with [Head of line
 * blocking](http://en.wikipedia.org/wiki/Head-of-line_blocking). For this
 * reason RPC traffic should only be used on localhost, where packet loss is
 * unlikely. If you want cross-host RPC you should probably be using E1.31
 * (sACN) or the Web APIs.
 *
 * The OLA client libraries follow the thin, asynchronous client model.
 * There is usually a 1:1 mapping between API methods and the underlying RPC
 * calls. The clients take the arguments supplied to the API method, construct
 * a request protobuf and invoke the stub method.
 *
 * The exception is the Java library, which could do with some love.
 *
 * The Python client continues to use the service generator that comes with
 * Protocol Buffers. The C++ client and olad (the server component) has
 * switched to our own generator, which can be found in protoc/ . If you're
 * writing a client in a new language we suggest you build your own generator,
 * protoc/CppGenerator.h is a good starting point.
 *
 * If you don't know C++ don't worry, we're happy to help. Once you know what
 * your generated code needs to look like we can build the generator for you.
 *
 * @section rpc_RPCLayer RPC Layer
 *
 * To make the RPC system message-agnostic we wrap the serialized request /
 * response protobufs in an outer layer protobuf. This outer layer also
 * contains the RPC control information, like what method to invoke, the
 * sequence number etc.
 *
 * The outer layer is defined in common/rpc/Rpc.proto and looks like:
  ~~~~~~~~~~~~~~~~~~~~~
  enum Type {
    REQUEST = 1;
    RESPONSE = 2;
    ...
  };

  message RpcMessage {
    required Type type = 1;
    optional uint32 id = 2;
    optional string name = 3;
    optional string name = 3;
    optional bytes buffer = 4;
  }
  ~~~~~~~~~~~~~~~~~~~~~
 *
 * The fields are as follows:
 * <dl><dt>type</dt>
 * <dd>The type of this RPC message, usually REQUEST or RESPONSE</dd>
 * <dt>id</dt>
 * <dd>the sequence number of the message. The sequence number of response messages
 * must match the number used in the request.</dd>
 * <dt>name</dt>
 * <dd>the name of the remote method to call</dd>
 * <dt>buffer</dt>
 * <dd>the serialized inner protocol buffer.</dd>
 * </dl>
 *
 * So putting it all together, the RpcChannel::CallMethod() should look
 * something like:
  ~~~~~~~~~~~~~~~~~~~~~
  void RpcChannel::CallMethod(const MethodDescriptor *method,
                              RpcController *controller,
                              const Message *request,
                              Message *reply,
                              SingleUseCallback0<void> *done) {
    RpcMessage message;
    message.set_type(REQUEST);
    message.set_id(m_sequence.Next());
    message.set_name(method->name());

    string output;
    request->SerializeToString(&output);
    message.set_buffer(output);

    // Send the message over the TCP connection.
    return SendMsg(&message);
  }
  ~~~~~~~~~~~~~~~~~~~~~
 *
 * That's it on the sender side at a high level, although there is a bit of
 * detail that was skipped over. For instance you'll need to store the sequence
 * number, the reply message, the RpcController and the callback so you can do
 * the right thing when the response arrives.
 *
 * @section rpc_RPCHeader RPC Header
 *
 * Finally, since protobufs don't contain length information, we prepend a
 * 4 byte header to each message. The first 4 bits of the header is a version
 * number (currently 1) while the remaining 28 bits is the length of the
 * serialized RpcMessage.
 *
 * \image html rpc-message.png "The wire format of a single RPC message"
 *
 * When writing the code that sends a message, it's very important to enable
 * TCP_NODELAY on the socket and write the 4 byte header and the serialized
 * RpcMessage in a single call to write(). Otherwise, the write-write-read
 * pattern can introduce delays of up to 500ms to the RPCs, <a 
 * href="http://en.wikipedia.org/wiki/Nagle's_algorithm">Nagle's algorithm</a>
 * has a detailed analysis of why this is so. Fixing this in the C++ & Python
 * clients led to a 1000x improvement in the RPC latency and a 4x speedup in
 * the RDM Responder Tests, see commit
 * a80ce0ee1e714ffa3c036b14dc30cc0141c13363.
 *
 * @section rpc_OLAServices OLA Methods
 *
 * The methods exported by olad are defined in common/protocol/Ola.proto . If
 * you compare the RPC methods to those provided by the ola::client::OlaClient
 * class (the C++ API) you'll notice they are very similar.
 *
 * @section rpc_NewClient Writing a new Client
 *
 * To write a client in a new language you'll need the following:
 *   - An implementation of Protocol Buffers for your language. Luckily there
 *     are <a href="https://code.google.com/p/protobuf/wiki/ThirdPartyAddOns">
 *     many implementations</a> available. It doesn't have to have a service
 *     generator (see the first point below).
 *   - A sockets library.
 *   - If you're writing an asynchronous client (and we recommend you do) you'll
 *     either need non-blocking I/O or a threading library.
 *
 * I'd tackle the problem in the following steps:
 *   - Write (or ask us to help write) the service generator for your language.
 *     This will allow you to generate the OlaServerService_Stub.
 *   - Write an implementation of RpcChannel and RpcController.
 *   - Write the code to establish a connection to localhost:9010 and read from
 *     the network socket when new data arrives.
 *
 * Once that's complete you should have a working RPC implementation. However
 * it'll still require the callers to deal with protobufs, stub objects and the
 * network code. The final step is to write the thin layer on top that presents
 * a clean, programmer friendly API, and handles the protobuf creation
 * internally.
 *
 * The last part is the least technically challenging, but it does require good
 * API design so that new methods and arguments can be added later.
 *
 * @section rpc_SimpleExample A Very Simple Example
 *
 * A quick example in Python style pseudo code. This constructs a DmxData
 * message, sends it to the server and the extracts the Ack response.
  ~~~~~~~~~~~~~~~~~~~~~
  # connect to socket
  server = socket.connect("127.0.0.1", 9010)

  # Build the request
  Ola_pb2.DmxData dmx_data
  dmx_data.universe = 1
  dmx_data.data = .....

  # Wrap in the outer layer
  Rpc_pb2.RpcMessage rpc_message
  rpc_message.type = RpcMessage::REQUEST
  rpc_message.id = 1
  rpc_message.name = 'UpdateDmxData'
  rpc_message.buffer = dmx_data.Serialize()

  server.write(rpc_message.Serialize())

  # wait for response on socket
  response_data = server.recv(1000)
  header = struct.unpack('<L', response_data)[0]
  # grab the first 4 bytes which is the header
  version, size = ((header & 0xf0000000) >> 28, header & 0x0ffffff)

  if version != 1:
    # Bad reply!
    return
  response = Rpc_pb2.RpcMessage()
  response.ParseFromString(response_data[4:size])
  if message.type != RpcMessage::RESPONSE:
    # not a response
    return
  if message.id != 1:
    # not the response we're looking for
    return

  Ola_pb2.Ack ack
  ack.ParseFromString(response.buffer)
  ~~~~~~~~~~~~~~~~~~~~~
 *
 */