1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364
|
.. highlight:: console
Sphinx Developer's Guide
========================
.. topic:: Abstract
This document describes the development process of Sphinx, a documentation
system used by developers to document systems used by other developers to
develop other systems that may also be documented using Sphinx.
.. contents::
:local:
The Sphinx source code is managed using Git and is hosted on GitHub.
git clone git://github.com/sphinx-doc/sphinx
.. rubric:: Community
sphinx-users <sphinx-users@googlegroups.com>
Mailing list for user support.
sphinx-dev <sphinx-dev@googlegroups.com>
Mailing list for development related discussions.
#sphinx-doc on irc.freenode.net
IRC channel for development questions and user support.
Bug Reports and Feature Requests
--------------------------------
If you have encountered a problem with Sphinx or have an idea for a new
feature, please submit it to the `issue tracker`_ on GitHub or discuss it
on the sphinx-dev mailing list.
For bug reports, please include the output produced during the build process
and also the log file Sphinx creates after it encounters an un-handled
exception. The location of this file should be shown towards the end of the
error message.
Including or providing a link to the source files involved may help us fix the
issue. If possible, try to create a minimal project that produces the error
and post that instead.
.. _`issue tracker`: https://github.com/sphinx-doc/sphinx/issues
Contributing to Sphinx
----------------------
The recommended way for new contributors to submit code to Sphinx is to fork
the repository on GitHub and then submit a pull request after
committing the changes. The pull request will then need to be approved by one
of the core developers before it is merged into the main repository.
#. Check for open issues or open a fresh issue to start a discussion around a
feature idea or a bug.
#. If you feel uncomfortable or uncertain about an issue or your changes, feel
free to email sphinx-dev@googlegroups.com.
#. Fork `the repository`_ on GitHub to start making your changes to the
**master** branch for next major version, or **stable** branch for next
minor version.
#. Write a test which shows that the bug was fixed or that the feature works
as expected.
#. Send a pull request and bug the maintainer until it gets merged and
published. Make sure to add yourself to AUTHORS_ and the change to
CHANGES_.
.. _`the repository`: https://github.com/sphinx-doc/sphinx
.. _AUTHORS: https://github.com/sphinx-doc/sphinx/blob/master/AUTHORS
.. _CHANGES: https://github.com/sphinx-doc/sphinx/blob/master/CHANGES
Getting Started
~~~~~~~~~~~~~~~
These are the basic steps needed to start developing on Sphinx.
#. Create an account on GitHub.
#. Fork the main Sphinx repository (`sphinx-doc/sphinx
<https://github.com/sphinx-doc/sphinx>`_) using the GitHub interface.
#. Clone the forked repository to your machine. ::
git clone https://github.com/USERNAME/sphinx
cd sphinx
#. Checkout the appropriate branch.
For changes that should be included in the next minor release (namely bug
fixes), use the ``stable`` branch. ::
git checkout stable
For new features or other substantial changes that should wait until the
next major release, use the ``master`` branch.
#. Optional: setup a virtual environment. ::
virtualenv ~/sphinxenv
. ~/sphinxenv/bin/activate
pip install -e .
#. Create a new working branch. Choose any name you like. ::
git checkout -b feature-xyz
#. Hack, hack, hack.
For tips on working with the code, see the `Coding Guide`_.
#. Test, test, test. Possible steps:
* Run the unit tests::
pip install -r test-reqs.txt
make test
* Again, it's useful to turn on deprecation warnings on so they're shown in
the test output::
PYTHONWARNINGS=all make test
* Build the documentation and check the output for different builders::
cd doc
make clean html latexpdf
* Run code style checks and type checks (type checks require mypy)::
make style-check
make type-check
* Run the unit tests under different Python environments using
:program:`tox`::
pip install tox
tox -v
* Add a new unit test in the ``tests`` directory if you can.
* For bug fixes, first add a test that fails without your changes and passes
after they are applied.
* Tests that need a sphinx-build run should be integrated in one of the
existing test modules if possible. New tests that to ``@with_app`` and
then ``build_all`` for a few assertions are not good since *the test suite
should not take more than a minute to run*.
#. Please add a bullet point to :file:`CHANGES` if the fix or feature is not
trivial (small doc updates, typo fixes). Then commit::
git commit -m '#42: Add useful new feature that does this.'
GitHub recognizes certain phrases that can be used to automatically
update the issue tracker.
For example::
git commit -m 'Closes #42: Fix invalid markup in docstring of Foo.bar.'
would close issue #42.
#. Push changes in the branch to your forked repository on GitHub. ::
git push origin feature-xyz
#. Submit a pull request from your branch to the respective branch (``master``
or ``stable``) on ``sphinx-doc/sphinx`` using the GitHub interface.
#. Wait for a core developer to review your changes.
Core Developers
~~~~~~~~~~~~~~~
The core developers of Sphinx have write access to the main repository. They
can commit changes, accept/reject pull requests, and manage items on the issue
tracker.
You do not need to be a core developer or have write access to be involved in
the development of Sphinx. You can submit patches or create pull requests
from forked repositories and have a core developer add the changes for you.
The following are some general guidelines for core developers:
* Questionable or extensive changes should be submitted as a pull request
instead of being committed directly to the main repository. The pull
request should be reviewed by another core developer before it is merged.
* Trivial changes can be committed directly but be sure to keep the repository
in a good working state and that all tests pass before pushing your changes.
* When committing code written by someone else, please attribute the original
author in the commit message and any relevant :file:`CHANGES` entry.
Locale updates
~~~~~~~~~~~~~~
The parts of messages in Sphinx that go into builds are translated into several
locales. The translations are kept as gettext ``.po`` files translated from the
master template ``sphinx/locale/sphinx.pot``.
Sphinx uses `Babel <http://babel.pocoo.org/en/latest/>`_ to extract messages
and maintain the catalog files. It is integrated in ``setup.py``:
* Use ``python setup.py extract_messages`` to update the ``.pot`` template.
* Use ``python setup.py update_catalog`` to update all existing language
catalogs in ``sphinx/locale/*/LC_MESSAGES`` with the current messages in the
template file.
* Use ``python setup.py compile_catalog`` to compile the ``.po`` files to binary
``.mo`` files and ``.js`` files.
When an updated ``.po`` file is submitted, run compile_catalog to commit both
the source and the compiled catalogs.
When a new locale is submitted, add a new directory with the ISO 639-1 language
identifier and put ``sphinx.po`` in there. Don't forget to update the possible
values for :confval:`language` in ``doc/config.rst``.
The Sphinx core messages can also be translated on `Transifex
<https://www.transifex.com/>`_. There exists a client tool named ``tx`` in the
Python package "transifex_client", which can be used to pull translations in
``.po`` format from Transifex. To do this, go to ``sphinx/locale`` and then run
``tx pull -f -l LANG`` where LANG is an existing language identifier. It is
good practice to run ``python setup.py update_catalog`` afterwards to make sure
the ``.po`` file has the canonical Babel formatting.
Coding Guide
------------
* Try to use the same code style as used in the rest of the project. See the
`Pocoo Styleguide`__ for more information.
__ http://flask.pocoo.org/docs/styleguide/
* For non-trivial changes, please update the :file:`CHANGES` file. If your
changes alter existing behavior, please document this.
* New features should be documented. Include examples and use cases where
appropriate. If possible, include a sample that is displayed in the
generated output.
* When adding a new configuration variable, be sure to document it and update
:file:`sphinx/quickstart.py` if it's important enough.
* Use the included :program:`utils/check_sources.py` script to check for
common formatting issues (trailing whitespace, lengthy lines, etc).
* Add appropriate unit tests.
Debugging Tips
~~~~~~~~~~~~~~
* Delete the build cache before building documents if you make changes in the
code by running the command ``make clean`` or using the
:option:`sphinx-build -E` option.
* Use the :option:`sphinx-build -P` option to run Pdb on exceptions.
* Use ``node.pformat()`` and ``node.asdom().toxml()`` to generate a printable
representation of the document structure.
* Set the configuration variable :confval:`keep_warnings` to ``True`` so
warnings will be displayed in the generated output.
* Set the configuration variable :confval:`nitpicky` to ``True`` so that Sphinx
will complain about references without a known target.
* Set the debugging options in the `Docutils configuration file
<http://docutils.sourceforge.net/docs/user/config.html>`_.
* JavaScript stemming algorithms in ``sphinx/search/*.py`` (except ``en.py``) are
generated by this
`modified snowballcode generator <https://github.com/shibukawa/snowball>`_.
Generated `JSX <https://jsx.github.io/>`_ files are
in `this repository <https://github.com/shibukawa/snowball-stemmer.jsx>`_.
You can get the resulting JavaScript files using the following command::
$ npm install
$ node_modules/.bin/grunt build # -> dest/*.global.js
Deprecating a feature
---------------------
There are a couple reasons that code in Sphinx might be deprecated:
* If a feature has been improved or modified in a backwards-incompatible way,
the old feature or behavior will be deprecated.
* Sometimes Sphinx will include a backport of a Python library that's not
included in a version of Python that Sphinx currently supports. When Sphinx
no longer needs to support the older version of Python that doesn't include
the library, the library will be deprecated in Sphinx.
As the :ref:`deprecation-policy` describes,
the first release of Sphinx that deprecates a feature (``A.B``) should raise a
``RemovedInSphinxXXWarning`` (where XX is the Sphinx version where the feature
will be removed) when the deprecated feature is invoked. Assuming we have good
test coverage, these warnings are converted to errors when running the test
suite with warnings enabled: ``python -Wall tests/run.py``. Thus, when adding
a ``RemovedInSphinxXXWarning`` you need to eliminate or silence any warnings
generated when running the tests.
.. _deprecation-policy:
Deprecation policy
------------------
A feature release may deprecate certain features from previous releases. If a
feature is deprecated in feature release 1.A, it will continue to work in all
1.A.x versions (for all versions of x) but raise warnings. Deprecated features
will be removed in the first 1.B release, or 1.B.1 for features deprecated in
the last 1.A.x feature release to ensure deprecations are done over at least 2
feature releases.
So, for example, if we decided to start the deprecation of a function in
Sphinx 1.4:
* Sphinx 1.4.x will contain a backwards-compatible replica of the function
which will raise a ``RemovedInSphinx16Warning``.
* Sphinx 1.5 (the version that follows 1.4) will still contain the
backwards-compatible replica.
* Sphinx 1.6 will remove the feature outright.
The warnings are displayed by default. You can turn off display of these
warnings with:
* ``PYTHONWARNINGS= make html`` (Linux/Mac)
* ``export PYTHONWARNINGS=`` and do ``make html`` (Linux/Mac)
* ``set PYTHONWARNINGS=`` and do ``make html`` (Windows)
Unit Testing
------------
Sphinx has been tested with pytest runner. Sphinx developers write unit tests
using pytest notation. Utility functions and pytest fixtures for testing are
provided in ``sphinx.testing``. If you are a developer of Sphinx extensions,
you can write unit tests with using pytest. At this time, ``sphinx.testing``
will help your test implementation.
How to use pytest fixtures that are provided by ``sphinx.testing``?
You can require ``'sphinx.testing.fixtures'`` in your test modules or
``conftest.py`` files like this::
pytest_plugins = 'sphinx.testing.fixtures'
If you want to know more detailed usage, please refer to ``tests/conftest.py``
and other ``test_*.py`` files under ``tests`` directory.
.. note::
Prior to Sphinx - 1.5.2, Sphinx was running the test with nose.
.. versionadded:: 1.6
``sphinx.testing`` as a experimental.
|