# Playground Logger Format

This document describes the format used by PlaygroundLogger to serialize
objects into a stream of bytes for the purpose of storage or transmission of such
objects.

It does not deal with validation of stored data, nor does it indicate the
preferred manner to display stored data, nor does it deal with the problem of
reconstructing object state from stored data.

## Primitive types

These are the basic storage types that will be used throughout this documentation:

* The `8bytes` type. A `8bytes` is an unsigned integer of 64 bits in size.

* The `number` type. A `number` is an unsigned integer of 64 bits in size.
  It can be stored in one of two forms:

    - short number. A number between 0 and 254 will be represented as itself
      on one byte of storage.
    - long number. A number 255 and larger will be represented as a byte with
      the value 255, plus the little-endian representation of the number in
      eight bytes.

  The underlying assumption is that the log will mostly trade in small integer
  values, which means that using one extra byte for long integers will be
  offset by the abundance of numbers that will instead fit in one byte.
  Practical experience may prove this wrong, in which case there are several
  other avenues worth pursuing for a compact representation of numbers.

* The `string` type. A `string` is a `number` length + that many bytes
  encoded as UTF8 text.

* The `utf8bytes` type. A `utf8bytes` is a sequence of bytes encoded as UTF8.
  This kind of data has no associated length, nor end-of-text marker. As such its
  main use case is being the payload of an IDERepr, in which the payload length
  serves the purpose of indicating the data size.

  UTF8 is a nice and compact encoding, which optimizes nicely in the presumably
  common case of ASCII-like strings.

* The `boolean` type. A `boolean` is a one-byte value where 0 means false and 1 means true.
  Other values are undefined and consumers should reject them.

* The `double` type. A `double` is an IEE-754 double precision binary floating point value.
   The data occupies 8 bytes.

* The `float` type. A `float` is an IEE-754 single precision binary floating point value.
   The data occupies 4 bytes.

## Header

A log always starts with a `number` which is the version number of the format
as used in this particular log entry. Different formats might add or remove
fields compared to what is described here.

The current value for the version number is 10.

Values of 0 and 1 are reserved.
Any value less than the current version number is deprecated.
Any value greater than the current version number is reserved.

This is followed by four `8bytes` values. These are meant to represent
two pairs:

* (starting line, starting column)

* (ending line, ending column)

For consumers that produce logging data by instrumenting source code, these
are the locations in the instrumented source code that are responsible for
generating the data being logged. No sanity checking of any kind is performed,
and consumers should be ready to receive any possible value combinations.

This is followed by a `number` field. This is the count of a set of
(`string`,`string`) pairs that provide status and configuration details
about the log being emitted. As of now, there is only one field that can
be emitted:

* "tid", a unique identifier for the thread that is emitting the log entry

Consumers should be ready and willing to accept any (or no) entries here and proceed.

Immediately following this header, there is a log entry for the logged object.

## Log Entry

A log entry starts with a `string` field giving the name of the object being
logged.
Consumers are required to accept any empty or non-empty string for this field.

This is immediately followed by a one-byte code which represents the type of
log entry. Values 0 and 255 are reserved, and should not be used. The values
currently in use are:

* 1, for `class`
* 2, for `struct`
* 3, for `tuple`
* 4, for `enum`
* 5, for `aggregate`
* 6, for `container`
* 7, for `iderepr`
* 8, for `gap`
* 9, for `scope_entry`
* 10, for `scope_exit`
* 11, for `error`
* 12, for `index_container`
* 13, for `key_container`
* 14, for `membership_container`

The values that follow this code depend on the type.

## Structured Entries

Entries of types:

* `class`
* `struct`
* `tuple`
* `enum`
* `aggregate`
* `container`
* `index_container`
* `key_container`
* `membership_container`

all follow the `structured` format.

A `structured` log entry is meant for objects that contain other objects.

A `structured` log entry contains:

* A `string` to be used as the type name.

  The exact contents depend on the individual object being logged, but in
  general this is going to be a typename.
  An empty string is allowed here.

* A `string` to be used as a summary.

  The exact contents depend on the individual object being logged, but in
  general this is expected to be a count of some sort, or a longer more
  detailed description of this object.
  An empty string is allowed here.

* A `number` that specifies the number of elements contained within the
  structured object (total-count).

  This count is non-recursive, meaning that a structured could contain other
  structured values within itself, and this will only be a first-level deep
  count rather than a total.

* A `number` that specifies the number of elements contained within the
  structured object **that are contained in the log** (stored-count).

  Due to the presence of gap elements, consumers are encouraged to avoid making deductions based on the relationship between `total-count` and `stored-count`.
  
  One exception is that the logger itself will, if the `total-count` field happens to be zero, not emit the `stored-count` field. It is then safe to assume that `total-count == 0 ==> stored-count == 0`. The rationale should be obvious.

Following this specification, there are as many log entries as specified to be
contained in the log. Each of them follows the general format of a log entry,
meaning they will each have a type marker, and they might well be themselves
`structured` entries.

## IDE Representation

An `iderepr` log entry is intended as a leaf, i.e. an object that does not
contain other objects. The exact interpretation of any given `iderepr` however
is largely up to the decoder, and beyond the scope of this discussion.

An `iderepr` log entry contains:

* A `boolean` that indicates whether the summary is the preferred sidebar display.

  The interpretation of this flag is left to consumers, but as a general guideline
  if this value is true, consumers should prefer showing the brief summary as the
  compact representation of data.

* A `string` to be used as the type name.

  The exact contents depend on the individual object being logged, but in
  general this is going to be a typename.
  An empty string is allowed here.

* A `string` to be used as a summary.

  The exact contents depend on the individual object being logged, but in
  general this is expected to be a count of some sort, or a longer more
  detailed description of this object.
  An empty string is allowed here.

* A `string` describing the semantics of the stored data.

  For instance, this field could say "HTML" to mean the data should be
  displayed as an HTML page, or "PNG" to mean the data should be displayed
  as a PNG format image.

  Encoders and decoders have to be in sync for this mechanism to work, of
  course, but it is assumed that this is a more extensible scheme than having
  a specification of a small set of representations forcing producers and
  consumer to all have to make do with those instead of having the freedom to
  invent new formats at will.

* A `number` giving the size in bytes of the actual logged data.

  This count does not include any of the format overhead, and should merely
  be considered the number of bytes to read to obtain the encoded object.

  The exact meaning of the bytes is dependent on the `string` field.
  This is a (potentially incomplete and not authoritative) list of formats:

    - STRN: A `utf8bytes` field.
    - SINT: A signed number (of 64 bits in size) represented as a `utf8bytes`
    - UINT: An unsigned number (of 64 bits in size) represented as a `utf8bytes`
    - FLOT: A single precision floating-point number represented as a `float`
    - DOBL: A single precision floating-point number represented as a `double`
    - IMAG: An image.
    - VIEW: A pictorial representation of a view.
    - SKIT: A pictorial representation of a software sprite.
    - COLR: An NSColor represented as the output of an NSKeyedArchiver
      with two objects inside of it:
      `IDEColorSpaceKey`: the name of color space as an NSString
      `IDEColorComponentsKey`: an NSArray of CGFloat NSNumbers that
      represent the components of the color in the given color space
    - BEZP: An NSKeyedArchiver-encoded NSBezierPath under the key "root"
    - ASTR: An NSKeyedArchiver-encoded NSAttributedString under the key "root"
    - PONT: A Point represented as an NSArchiver-encoded pair of double values
      under the keys "x" and "y"
    - SIZE: A Size represented as an NSArchiver-encoded pair of double values
      under the keys "w" and "h"
    - RECT: A Rectangle represented as an NSArchiver-encoded set of double values
      under the keys "x", "y", "w" and "h"
    - RANG: A Range represented as an NSArchiver-encoded pair of unsigned 64-bit
      integer values under the keys "loc" and "len"
    - BOOL: A boolean represented as a one-byte value, where
      1 == true
      anything else == false
    - URL: A `utf8bytes` field representing a Uniform Resource Locator.

## Gap

A `gap` log entry is intended as a placeholder for things that are missing
from the log as a result of a policy decision on how many elements to log.

A `gap` entry contains no data. Upon reading the marker, consumers are done
decoding the entire content of the entry.

It is up to consumers to decide how to visualize a `gap` entry.

It should be noted that the `name` field won't mean a lot for a `gap` entry.
However, consumers should be capable of decoding a string there - including an
empty one.

## Scope Entry

A `scope` entry (`scope_entry`, `scope_exit`) is intended to let the consumer
know when a certain scope has been "entered" or "exited". The concepts of
scope, entry and exit are left for the language to define, and it is assumed
the reader of this document is familiar enough with the language to have an
understanding of such concepts.

A `scope` entry contains no data. Upon reading the marker, consumers are done
decoding the entire content of the entry. The underlying assumption is that
some other layer of communication is providing location or addressing information
to enable the consumer to follow through the flow of entry and exit log entries.

It is up to consumers to decide how to visualize a `scope` entry.

It should be noted that the `name` field won't mean a lot for a `scope` entry.
However, consumers should be capable of decoding a string there - including an
empty one.

## Error

An `error` log entry is intended as a placeholder for things that are missing
from the log as a result of an internal logger issue with producing data.

An `error` entry is followed by `string` data, possibly an empty `string`.
This is meant to provide an explanation for the issue - and will most likely
be empty when the logger doesn't have an explanation (e.g. runtime errors).

It is up to consumers to decide how to visualize an `error` entry.