.. include:: header.rst

=====================================================
EP 10 — Public API and Backwards Compatibility Policy
=====================================================

:Authors: Günter Milde, Adam Turner
:Discussions-To: https://sourceforge.net/p/docutils/feature-requests/89/
:Status: Draft
:Type: Process
:Created: 2025-04-22
:Docutils-Version: 1.0
:Abstract:
  This document suggests a definition of the public APIs provided by the
  Docutils project and the backwards compatibility policy.

.. contents::


Motivation
==========

.. Clearly explain why the existing specification is inadequate to
   address the problem that the proposal solves.

Docutils has a large user base and is used in production at several
places (Python documentation, Linux kernel documentation, CMake
documentation, readthedocs, ...). OTOH, Docutils has a version number
below 1.0 (widely seen as an indicator of "beta" status of a project).

The current `Docutils Project Policies`_ section on `version
identifcation`_ concentrates on the formal definition of the version
specifier but leaves open what consists a "major change in the design
or API".

The current `backwards compatibility policy`_ is a stub referencing
:PEP:`387`.


Rationale
=========

.. Describe why particular design decisions were made.

Clearly defining how we will balance evolution with stability is
important to both users and project developers.

People affected by changes in Docutils include:

.. class:: description

Authors
  writing or maintaining reStructuredText documents.

End-Users
  of Docutils native `front-end tools`_ (optionally with 3rd-party
  drop-in extensions) or alternative tools using Docutils either as a
  library (Sphinx_, …) or via the command line interface
  (build systems, Makefiles, scripts in other languages).

Developers
  i.e. authors and maintainers of

  - projects that use Docutils as a library (Sphinx_, rsted_, Leo_,
    Pelican_, ebookmaker_, MyST_, readthedocs_, rinohtype_, …),
  - drop-in components (pycmark_, rst2pdf_, rst2beamer_, …),
  - alternative front-end tools,
  - custom stylesheets (CSS style sheets, LaTeX styles, ODT styles),
    or
  - re-implementations of the `reStructuredText specification`_,
    e.g. Pandoc_ or Text-Restructured_ (prest).

A person may belong to more than one of these catgories.

.. _Sphinx: https://www.sphinx-doc.org/
.. _rsted: https://github.com/anru/rsted
.. _Leo: https://leo-editor.github.io/leo-editor/
.. _Pelican: https://docs.getpelican.com/en/latest/
.. _MyST: https://myst-parser.readthedocs.io/
.. _readthedocs: https://readthedocs.org/
.. _pycmark: https://github.com/pycmark/pycmark
.. _rst2pdf: https://rst2pdf.org/
.. _rst2beamer: https://docutils.sourceforge.io/sandbox/rst2beamer/
.. _rinohtype: https://pypi.org/project/rinohtype/
.. _ebookmaker: https://pypi.org/project/ebookmaker/
.. _PanDoc: https://pandoc.org/
.. _Text-Restructured: https://metacpan.org/dist/Text-Restructured


Specification
=============

.. Describe the syntax and semantics of any new feature.

Docutils public APIs are:

* the `reStructuredText specification`_,

* the `Docutils document structure`_ (`Docutils Document Tree`),

* names, command-line arguments and behaviour of
  the `"console_scripts" entry points`_ (`Front-end Tools`),

* the core ``docutils`` Python package API:

  - the `Docutils Publisher`_ interface for programmatic use,

  - component interfaces as defined by the abstract base classes
    `docutils.reader.Reader`, `docutils.writer.Writer`, and
    `docutils.transform.Transform`,

* behaviour and names of all *documented Python objects*, [#]_

* *output templates* and *style sheets* provided with the writers,

  .. template.txt, default.tex, titlepage.tex, xelatex.tex
  .. html4css1.css, minimal.css, docutils.sty (LaTeX), styles.odt

* the interface to custom stylesheets -- elements, macros and classes
  used by writers to represent doctree_ nodes in the output format.

Exemptions:
  Python objects, stylesheets and templates can explicitly "opt-out" of
  the public API with a docstring noting that the object is provisional_
  or internal.

  All undocumented objects should be assumed to be internal. [#]_

See also the `API Reference Material for Client-Developers`_.

.. [#] Cf. `PEP 387: Backwards Compatibility Rules`_
.. [#] Cf. `PEP 008: Public and Internal Interfaces`_


Backwards Compatibility
=======================

.. Describe potential impact and severity on pre-existing code.

Beginning with version 1.0, Docutils will follow the rules of
`Semantic Versioning`_. All incompatible changes to the public APIs
require increasing the `major` part of the `version specifier`_.
Backwards compatible changes can be done in `minor` releases.


Security Implications
=====================

If required, critical bug fixes may change the public API without advance
warning.


How to Teach This
=================

.. How to teach users, new and experienced,
   how to apply the proposal to their work.

* Move the API specification_ and the backwards compatibility declaration
  to the `Docutils Project Policies`_.

* Complete the API documentation and keep it up to date.

* Generate "docutils" package API documentation from the docstrings:

  - Fix/enhance/add docstrings to improve the output of `pydoc`_.

  - Generate API documentation with Sphinx:

    - nicely format rST docstrings
    - include attribute docstrings (ignored by pydoc_).

* Put the following text at a suitable place in the documentation:

     To find out if an object from the "docutils" package is safe to use,
     look up its docstring and the docstring of its parent(s) [#]_.

     If there is no documentation or the documentation says "provisional" or
     "internal", the name, behaviour, and existence of the object is not
     guaranteed to be stable.

     Code relying on non-public objects should be made robust using
     public alternatives. If there is a no such alternative or the
     required change would be a problem, `contact the Docutils
     developers`__ or file a `feature request`_.

     .. [#] `Attribute docstrings`_ are not shown by pydoc_. To find out
        whether attributes have a docstring, check the source.

     .. _attribute docstrings:
         https://peps.python.org/pep-0258/#attribute-docstrings
     .. _pydoc: https://docs.python.org/3/library/pydoc.html
     __ https://docutils.sourceforge.io/docs/user/mailing-lists.html
     .. _feature request: https://sourceforge.net/p/docutils/feature-requests/


Rejected Ideas
==============

.. Why certain ideas that were brought while discussing this proposal were not
   ultimately pursued.

* Use type annotations as an indication of status in the public API.

  - There is no known precedence for this approach.
  - Type annotations may be helpful also for non-public code.

* Use Calendar Versioning (CalVer).

  - Would be a break from current versioning without clear advantages.

* Allow breaking API changes in *minor* versions after prior announcement
  and a deprecation period.

  - Breaks the principle of least surprise.

* Enumerate all modules, classes, and functions that form the public API.

* Mark all private objects with a prefix underscore.

  - May needlessly break applications that use "internal" objects by the
    current name.
  - Too much work.


Open Issues
===========

.. Any points that are still being decided/discussed.

* Differentiate between "core API" and "extended API"?

  Cf. the `Docutils Project Policies`_

    When Docutils reaches version 1.0, the major APIs will be considered
    frozen.

    The major number [...] may be incremented later if there is a major
    change in the design or API.


* Formalise the wording for docstrings for public/private/provisional
  (ideally this would be a single regex pattern)?

  * The keyword provisional_ is well defined. ✓
  * Use "private" or "internal"?

* Declare only objects included in the ``__all__`` attribute of their
  parent objects as public rsp. explicitely list all public objects in
  ``__all__`` attribute of their parents?

  This would hide private objects from `pydoc` help on the parent objects.

* Define a minimum deprecation time similar to Docbook__? E.g.

    * A "major" release may contain backward-incompatible changes if:

      * the change was announced in the release notes for the previous
        version (major or minor) and
      * the change was announced in a release that occurred at least six
        months previously.

    By these rules, Docutils developers can announce, in release 5.1, for
    example, its plans to make a backward-incompatible change in release 6.0.
    Then, in 6.0, if it’s been at least six months since 5.1 was
    released, they can make that change.

  __ https://tdg.docbook.org/tdg/5.1/ch01.html#bwcompat


References
==========

.. A collection of URLs used as references through the proposal.

.. _API Reference Material for Client-Developers:
    ../index.html#api-reference-material-for-client-developers
.. _doctree:
.. _Docutils document structure: ../ref/doctree.html
.. _docutils.dtd: ../ref/docutils.dtd
.. _Docutils Design Specification: ../peps/pep-0258.html
.. _Docutils Project Policies: ../dev/policies.html
.. _version specifier:
.. _version identifcation:
    ../dev/policies.html#version-identification
.. _backwards compatibility policy:
    ../dev/policies.html#backwards-compatibility-policy
.. _"console_scripts" entry points:
.. _front-end tools: ../user/tools.html
.. _Docutils Publisher: ../api/publisher.html
.. _Docutils Transforms: ../ref/transforms.html
.. _HISTORY: ../docutils/HISTORY.html
.. _RELEASE-NOTES: ../docutils/RELEASE-NOTES.html
.. _reStructuredText specification:
    ../ref/rst/restructuredtext.html

.. _`PEP 387: backwards compatibility rules`:
    https://peps.python.org/pep-0387/#backwards-compatibility-rules
.. _`PEP 008: Public and Internal Interfaces`:
    https://peps.python.org/pep-0008/#public-and-internal-interfaces
.. _provisional: https://docs.python.org/3/glossary.html#term-provisional-API
.. _Semantic Versioning: https://semver.org/


Copyright
=========

This document is placed in the public domain or under the
CC0-1.0-Universal license, whichever is more permissive.