Config file format
==================

Libmegapixels gets its camera pipeline definitions from config files stored in one of the config directories.
For finding config files libmegapixels will scan a few preconfigured path from the "compatible" names of the device.

The first part of this puzzle is the compatible names, these can be found at runtime from
the :code:`/proc/device-tree/compatible` file or otherwise from the top-most "compatible" node in the device tree.

As an example the Xiaomi Mi Note 2 will be used. The compatible list for the device tree for this device is at
https://github.com/torvalds/linux/blob/b85ea95d086471afb4ad062012a4d73cd328fa86/arch/arm64/boot/dts/qcom/msm8996pro-xiaomi-scorpio.dts#L17

The relevant line here is this:

.. code-block:: device-tree

   compatible = "xiaomi,scorpio", "qcom,msm8996pro", "qcom,msm8996";

The second part is the lookup paths. They are checked in this order:

* :code:`config/$model.conf` in the current working directory
* :code:`/etc/megapixels/config/$model.conf`
* :code:`/usr/share/megapixels/config/$model.conf`

So on this device the file will be looked up in this order:

* :code:`config/xiaomi,scorpio.conf`
* :code:`/etc/megapixels/config/xiaomi,scorpio.conf`
* :code:`/usr/share/megapixels/config/xiaomi,scorpio.conf`
* :code:`config/qcom,msm8996pro.conf`
* :code:`/etc/megapixels/config/qcom,msm8996pro.conf`
* :code:`/usr/share/megapixels/config/qcom,msm8996pro.conf`
* :code:`config/qcom,msm8996.conf`
* :code:`/etc/megapixels/config/qcom,msm8996.conf`
* :code:`/usr/share/megapixels/config/qcom,msm8996.conf`

The first matched file will be used.

The file format
---------------

The config files are parsed by `libconfig <https://hyperrealm.github.io/libconfig/libconfig_manual.html#Configuration-Files>`_
and define all the cameras present on the specific device and the modes that are possible on this hardware.

An example of a minimal config is this:

.. code-block:: config

  Version = 1;
  Make: "Xiaomi";
  Model: "Scorpio";

  Rear: {
      SensorDriver: "imx318";
      BridgeDriver: "qcom-camss";

      Modes: (
      {
  	Width: 3840;
  	Height: 2160;
  	Rate: 30;
  	Format: "RGGB10";
  	Rotate: 90;

  	Pipeline: (
  	    {Type: "Link", From: "imx318", FromPad: 0, To: "msm_csiphy0", ToPad: 0},
  	    {Type: "Link", From: "msm_csiphy0", FromPad: 1, To: "msm_csid0", ToPad: 0},
  	    {Type: "Link", From: "msm_csid0", FromPad: 1, To: "msm_ispif0", ToPad: 0},
  	    {Type: "Link", From: "msm_ispif0", FromPad: 1, To: "msm_vfe0_rdi0", ToPad: 0},
  	    {Type: "Mode", Entity: "imx318"},
  	    {Type: "Mode", Entity: "msm_csiphy0"},
  	    {Type: "Mode", Entity: "msm_csid0"},
  	    {Type: "Mode", Entity: "msm_ispif0"},
  	);
  	},
      );
  };

The only top-level keys are Version, Make and Model. The Version should always be 1 and is reserved for possible future
breaking updates.

The Make and Model keys set the human-readable name for the device which will be written in the EXIF data of pictures as
he camera model.

Camera definitions
------------------

All the other top-level keys in the config file are definitions for cameras. The name is not very important and is for
documentation purposes.

The keys in the camera block are:

* :code:`BridgeDriver`: The name of the driver that provides the /dev/media and /dev/video node for this camera
* :code:`SensorDriver`: The name of the sensor entity in the media graph
* :code:`FlashPath`: optional, defines the path to a LED device to use as the flash for this camera.
* :code:`FlashDisplay`: optional, defines that there is no LED that can be used as flash, use the display as the
  light-source instead.

The camera section requires a :code:`SensorDriver` and :code:`BridgeDriver` key. The BridgeDriver will identify the bridge device (eg.
/dev/video0) and media device (eg. /dev/media0). This combination should uniquely identify a single camera on the device.

These values can be found using the :code:`media-ctl` utility.

.. code-block:: shell-session

   $ media-ctl --device 0 --print-topology
   Media controller API version 6.1.14

   Media device information
   ------------------------
   driver          qcom-camss
   model           Qualcomm Camera Subsystem
   serial
   bus info        platform:a34000.camss
   hw revision     0x0
   driver version  6.1.14

   Device topology
   - entity 1: msm_csiphy0 (2 pads, 5 links)
               type V4L2 subdev subtype Unknown flags 0
               device node name /dev/v4l-subdev0
   	pad0: Sink
   		[fmt:UYVY8_2X8/1920x1080 field:none colorspace:srgb]
   		<- "imx318 3-001a":0 [ENABLED,IMMUTABLE]
   	pad1: Source
   		[fmt:UYVY8_2X8/1920x1080 field:none colorspace:srgb]
   		-> "msm_csid0":0 []
   		-> "msm_csid1":0 []
   		-> "msm_csid2":0 []
   		-> "msm_csid3":0 []

   [ Removed A LOT of entities here for brevity ]

   - entity 226: imx318 3-001a (1 pad, 1 link)
                 type V4L2 subdev subtype Sensor flags 0
                 device node name /dev/v4l-subdev19
   	pad0: Source
   		[fmt:SRGGB10_1X10/5488x4112@1/30 field:none colorspace:raw xfer:none]
   		-> "msm_csiphy0":0 [ENABLED,IMMUTABLE]

   - entity 228: ak7375 3-000c (0 pad, 0 link)
                 type V4L2 subdev subtype Lens flags 0
                 device node name /dev/v4l-subdev20

The BridgeDriver config node needs to match the driver listed at the top of the output. In this case :code:`qcom-camss`.

For the SensorDriver the entity needs to be found that represents the sensor. This is mentioned as
:code:`subtype Sensor` in the list and in the example is entity 226. The entity will be matched by the name listed
directly after the number which is :code:`imx318 3-001a`. The full name doesn't need to be used but is matched by prefix
to avoid the config needing to hardcode the i2c bus number for the sensor which might change on enabling/disabling
kernel modules.

Camera modes
------------

Every camera block requires to have at least one mode block but can have multiple. This is required since different
usecases of the camera need different modes. Usually the maximum resolution of the sensor will be used for taking
pictures but at that resolution the framerate will generally be too low for realtime preview.

For example the OV5640 sensor can run at 2592x1944 resolution for decently sharp pictures, but at that resolution is
limited to 15fps which will be quite annoying for the live-preview and not fast enough for smooth video recording. This
is why the PinePhone defines a 2592x1944@15fps mode and a 1280x720@30fps mode for the preview. The Megapixels application
will default to the highest defined resolution for taking pictures and will take the resolution closest to the display
resolution as the preview mode.

The required configuration keys for the mode are:

* :code:`Width` and :code:`Height` for the resolution of image _after_ any processing by the ISP.
* :code:`Rate` the frame interval in frames-per-second for this mode.
* :code:`Format` is the pixelformat for the data after ISP processing.

The optional keys are:

* :code:`Transfer` sets the transfer curve of the image data, can be "srgb" or "raw".
* :code:`Rotate` defines the rotation for matching up the orientation of the sensor and the display of the device.
* :code:`Mirror` can be set to :code:`true` to flip the image for the front-facing sensors.
* :code:`FocalLength` sets the effective focal length of the camera in this mode. Sensors generally crop the image
  slightly depending on the resolution which changes the focal length for the mode. This is only used for EXIF metadata
  and is defined in milimeters.
* :code:`FNumber` sets the aperture size of the lens. 3.0 will mean F/3 as aperture size, this is only used for EXIF

Mode pipeline
-------------

To make the mode actually work the pipeline has to be added. This is a small script that defines the ioctls that need to
be sent to the drivers to set up all the hardware in this mode. This is a full scripting system to deal with
inconsistencies in the Linux drivers, especially the staging drivers for sensors.

The pipeline script is a list of command objects. The specific command is set using the :code:`Type` key in this object.
All other keys in the command object depend on the specific command.

The possible commands are:

* `Link`_
* `Mode`_
* `Rate`_
* `Crop`_

The commands all accept the name of an entity from MediaCtl and these are prefix matched by default. This is because
for things like sensors there might be an i2c bus ID in the name of the entity and the bus ID is not stable across
reboots and kernel changes. There is one case though where the prefix match doesn't work when the name of one entity
is the exact prefix of one with a longer name. For this case prefix matching can be disabled with the `ExactName`
key in every command.

Cascading values
^^^^^^^^^^^^^^^^

In order to not repeat the values for resolutions, formats and framerates in every command values are cascaded
internally if nothing is specified. These cascading values are initialized to the resolution and mode for the Mode
block that contains this pipeline. If any command in the pipeline hardcodes another value then that new default will
also be used for the following commands. This means that if the pipeline script is defined in strict source->sink order
the minimum amount of repetitions of these values are required.

For example this pipeline:

.. code-block:: config

      Modes: (
          {
              Width: 4208;
              Height: 3120;
              Rate: 30;
              Format: "RGGB8";

              Pipeline: (
                  {Type: "Mode", Entity: "imx258", Format: "RGGB10P"},
                  {Type: "Mode", Entity: "rkisp1_csi"},
                  {Type: "Mode", Entity: "rkisp1_isp"},
                  {Type: "Mode", Entity: "rkisp1_isp", Pad: 2, Format: "RGGB8"},
                  {Type: "Crop", Entity: "rkisp1_isp"},
                  {Type: "Crop", Entity: "rkisp1_isp", Pad: 2},
                  {Type: "Mode", Entity: "rkisp1_resizer_mainpath"},
                  {Type: "Mode", Entity: "rkisp1_resizer_mainpath", Pad: 1}
              );
          },

The resolution and framerate is never defined in any of the pipeline commands and are inherited from the values set in
the mode. The pixelformat is specified twice here since the ISP will convert between RGGB10P and RGGB8 formats. Once to
set the format of the sensor and once on the ISP to return it to the value of the Mode block.

This pipeline will be interpreted as:

.. code-block:: config

   {Type: "Mode", Entity: "imx258", Width: 4208, Height: 3120, Format: "RGGB10P"},
   {Type: "Mode", Entity: "rkisp1_csi", Width: 4208, Height: 3120, Format: "RGGB10P"},
   {Type: "Mode", Entity: "rkisp1_isp", Width: 4208, Height: 3120, Format: "RGGB10P"},
   {Type: "Mode", Entity: "rkisp1_isp", Pad: 2, Width: 4208, Height: 3120, Format: "RGGB8"},
   {Type: "Crop", Entity: "rkisp1_isp", Width: 4208, Height: 3120, Top: 0, Left: 0},
   {Type: "Crop", Entity: "rkisp1_isp", Pad: 2, , Width: 4208, Height: 3120, Top: 0, Left: 0},
   {Type: "Mode", Entity: "rkisp1_resizer_mainpath", Width: 4208, Height: 3120, Format: "RGGB8"},
   {Type: "Mode", Entity: "rkisp1_resizer_mainpath", Pad: 1, Width: 4208, Height: 3120, Format: "RGGB8"}

Link
^^^^

The Link command is the most important one, this defines the media pipeline itself by specifying which links should be
created and broken. This is equivalent to the :code:`MEDIA_IOC_SETUP_LINK` ioctl.

Parameters:

* :code:`From` source entity name
* :code:`To` sink entity name
* :code:`FromPad` pad index on the source entity
* :code:`ToPad` pad index on the sink entity
* :code:`ExactName` match the exact From/To names instead of just a prefix.

Example:

.. code-block:: config

   {Type: "Link", From: "ov5640", FromPad: 0, To: "sun6i-csi-bridge", ToPad: 0},

Is equivalent to:

.. code-block:: shell-session

   $ media-ctl -l "'ov5640':0 -> 'sun6i-csi-bridge':0 [1]"

Mode
^^^^

The mode sets the V4L2 format on the entity pad, some drivers in V4L2 will cascade these values down the pipeline and
some will need the mode to be set on every entity in the pipeline to make the pipeline start streaming correctly.

The most minimal option is specifying just the entity name here and the rest of the values are used from the resolution
and format specified in the mode block this pipeline is in, or the values of the previous Mode command if one hardcodes
a value. This is for pipelines where an ISP changes the format or resolution halfway through the pipeline.

This command is equivalent to the :code:`VIDIOC_SUBDEV_S_FMT` ioctl.

Parameters:

* :code:`Entity` media entity to set the mode on
* :code:`Pad` pad index on the entity, defaults to 0
* :code:`Width` Resolution width, if not specified it will use value of the previous command
* :code:`Height` Resolution width, if not specified it will use value of the previous command
* :code:`Format` Pixelformat for the mode, if not specified it will use the value of the previous command
* :code:`SkipTry` disables the use of the VIDIOC_SUBDEV_S_FMT ioctl with V4L2_SUBDEV_FORMAT_TRY. This is for drivers
  that don't implement the V4L2_SUBDEV_FORMAT_TRY functionality or where it returns the wrong data.
* :code:`ExactName` match the exact Entity name instead of just a prefix.


Example:

.. code-block:: config

   {Type: "Mode", Entity: "ov5640", Pad: 0, Width: 640, Height: 480, Format: "BGGR8"},
   {Type: "Mode", Entity: "sun6i-csi-bridge"},

Is equivalent to:

.. code-block:: shell-session

   $ media-ctl --set-v4l2 '"ov5640":0 [fmt:SBBGR8_1X8/640x480]'
   $ media-ctl --set-v4l2 '"sun6i-csi-bridge":0 [fmt:SBBGR8_1X8/640x480]'

Rate
^^^^

The mode sets the framerate on the entity for the drivers that believe in such things, some drivers will require this
to be set and some will reject this ioctl altogether.

Just like the Mode command the rate value will cascade from the rate specified in the Mode block that contains this
pipeline and previous Rate calls that changes the pipeline framerate.

This command is equivalent to the :code:`VIDIOC_SUBDEV_S_FRAME_INTERVAL` ioctl.

Parameters:

* :code:`Entity` media entity to set the mode on
* :code:`Rate` frame interval, if not specified it will use value of the previous command
* :code:`ExactName` match the exact Entity name instead of just a prefix.


Example:

.. code-block:: config

   {Type: "Rate", Entity: "ov5640", Rate: 30},

Is equivalent to:

.. code-block:: shell-session

   // Only the @30 part here is relevant
   $ media-ctl --set-v4l2 '"ov5640":0 [fmt:SBBGR8_1X8/640x480@30]'


Crop
^^^^

Some drivers will allow cropping out a section of the frame, usually this is to remove dummy pixels from the edge of the
sensor. Some drivers will internally set the cropping automatically when setting the capture resolution and some will
fail to start until you specify the cropping to be 0 on all edges instead of having that as default. There are also
drivers that don't update the crop values when changing resolution so the crop ioctl needs to be called.

This command is equivalent to the :code:`VIDIOC_SUBDEV_S_CROP` ioctl.

Parameters:

* :code:`Entity` media entity to set the crop on
* :code:`Pad` pad index on the entity, defaults to 0
* :code:`Width` Crop width, if not specified it will use the mode width
* :code:`Height` Crop height, if not specified it will use the mode height
* :code:`Top` Top offset of the cropped region, defaults to 0
* :code:`Left` Left offset of the cropped region, defaults to 0
* :code:`ExactName` match the exact Entity name instead of just a prefix.

Example:

.. code-block:: config

   {Type: "Crop", Entity: "rkisp1_isp", Width: 640, Height: 480, Top: 0, Left: 0},

Is equivalent to:

.. code-block:: shell-session

   // Only the last crop: part here is relevant
   $ media-ctl --set-v4l2 '"rkisp1_isp":0 [fmt:SBBGR8_1X8/640x480 crop:(0,0)/640x480]'