1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
|
Internationalization
====================
This section covers how to internationalize (I18n) and localize (L10n) the PyData Sphinx Theme.
For details on how to localize the configurable strings in your Sphinx project, see the
:ref:`User guide section on internationalization <user-guide-i18n>`.
The PyData Sphinx Theme I18n/i10n workflows are based on `GNU Gettext <http://www.gnu.org/software/gettext/>`__
and `pybabel <https://babel.pocoo.org/en/latest/messages.html>`__ is used to keep the catalogs up to date.
Crowd-sourced localizations are managed on `Transifex <https://explore.transifex.com/12rambau/pydata-sphinx-theme/>`__.
.. note::
**Internationalization** (or I18n) is the process of making a program or application aware of and able to support multiple
languages.
**Localization** (L10n) is the process of translating localized programs or applications into different languages while
ensuring that the translations are correct for some native language and cultural habits.
The formal description of specific set of cultural habits for some country, together with all associated translations
targeted to the same native language, is called the *locale* for this language or country.
For more information about localization and internationalization see `GNU gettext concepts <https://www.gnu.org/software/gettext/manual/html_node/Concepts.html>`__.
The general process for internationalizing and localizing the theme is as follows:
#. :ref:`Mark strings in the theme as localizable <adding-localizable-text>`.
#. :ref:`Extract localizable strings <updating-localization-files>` to a message catalog template ``POT`` (``PO`` template file).
#. Generare a ``PO`` file from the ``POT`` file for :ref:`a new language (locale) you want to localize <adding-new-language>`
(or :ref:`update existing localization files <updating-localization-files>`).
#. :ref:`Compile the message catalogs <compiling-localization-files>` to binary ``MO`` files.
#. :ref:`Localize the theme <localizing-the-theme>`.
Localization files
-------------------
There are three types of files used in the localization process:
#. `PO files <https://www.gnu.org/software/gettext/manual/gettext.html#PO-Files>`__ ( Portable Object, also known as message catalogs)
associate each original, translatable string (defined in ``msgid``) of a given program with its translation in a
particular target language (defined in ``msgstr`` fields). A single PO file is dedicated to a single target language.
#. ``MO`` (Machine Object) files are a binary version of a PO file.
#. ``POT`` (Portable Object Template) files are similar to PO files, but with empty translations.
They are used as a template for new languages.
.. _adding-localizable-text:
Marking strings as localizable
------------------------------
All natural language text in the theme's components and layouts must be marked as localizable so that they can be later translated (or localized) into other languages.
For example, if you add a button with the text **Next page**, you will need to mark this text as localizable.
``HTML`` templates (located in the ``src/pydata_sphinx_theme/theme/`` directory).
To do so, you can use the Jinja2 ``trans`` block and/or a ``_()`` function to mark text as localizable in the corresponding
For example, to mark the text **Next page** as localizable, you would write:
.. code-block:: jinja
<button type="button">
{{- _("Next page") -}}
</button>
Any text that is marked in this way will be discoverable by ``pybabel`` and used to generate the localization files
(``PO`` files).
Once you've marked the text as localizable, complete the steps outlined in the :ref:`updating-localization-files`
section of this documentation.
For more details on marking strings as localizable in jinja templates visit `the Jinja2 documentation <https://jinja.palletsprojects.com/en/3.0.x/templates/>`__.
.. tip::
Remember to `manually escape variables <https://jinja.palletsprojects.com/en/2.11.x/templates/#working-with-manual-escaping>`__ if needed.
Sometimes, it can help localizers to describe where a string comes from or whether it refers to a noun or verb,
particularly if it can be difficult to find in the theme, or if the string itself is not very self-descriptive (for example, very short strings).
If you immediately precede the string with a comment that starts with ``L10n:``, the comment will be added to the PO
file, and visible to localizers. For example:
.. code-block:: jinja
{# L10n: Navigation button at the bottom of the page #}
<button type="button">
{{- _('Next page') -}}
</button>
.. _updating-localization-files:
Updating the localization files
-----------------------------------
When you add or change natural language text in the theme, you must update the message catalogs to include the new or
updated text. Follow these steps:
#. Edit the natural language text and ensure it is :ref:`marked as localizable <adding-localizable-text>`.
#. Extract the strings and update the localization files (``POT`` file):
.. code-block:: bash
# note this will by default update all the localization files for all the supported locales
tox run -e i18n-extract
This will update the localization files with new information about the position and text of the language you have modified.
If you *only* change non-localizable text (like HTML markup), the ``extract`` command will only update the
positions (line numbers) of the localizable strings.
Updating positions is optional - the line numbers are to inform the human translator, not to perform the localization.
But it is best practice to keep the positions up to date.
If you change localizable strings, the above command will extract the new or updated strings to localization template
file (``POT``) and perform a fuzzy match between the new or updated strings and existing localizations in the
localization files.
If there is a fuzzy match, a comment like ``#, fuzzy`` is added before the matched entry, this means that the
localization needs to be manually reviewed and possibly updated.
If after reviewing the localization you decide to keep the existing localization, you can remove the ``#, fuzzy``
comment from the entry.
If there is no fuzzy match, it will add a new localization entry.
You can learn more about fuzzy entries in the `GNU gettext manual <https://www.gnu.org/software/gettext/manual/gettext.html#Fuzzy-Entries>`__.
.. _compiling-localization-files:
Compiling the localization files
--------------------------------
Gettext doesn't parse any text files, it reads a binary format for faster performance. To compile the latest PO files in
the repository run:
.. code-block:: bash
tox run -e i18n-compile
You can also run the extract, update and compile commands in one go:
.. code-block:: bash
tox run -m i18n
This will update the localization files and compile them into binary ``MO`` files in a single step.
However, if there are fuzzy matches needing review, the compilation will fail, and you will need to review the
localizations manually.
Then compile the files again.
.. _adding-new-language:
Adding a new language
----------------------
The list of languages with existing (possibly incomplete) localizations is available in the
``src/pydata_sphinx_theme/locale`` directory.
To add a new language, follow these steps:
#. Identify the `ISO 639-1 code <https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes>`__ for the new language.
#. Generate a ``PO`` file based on ``POT`` file for this new language:
.. code-block:: bash
# for example, to add Quechua (ISO 639-1 code: qu)
tox -e i18n-new-locale -- qu
#. Update and compile the localization files as described in the :ref:`updating-localization-files` and
:ref:`compiling-localization-files` sections. Then commit the changes.
#. Localize the theme's into the newly added language (see :ref:`localizing-the-theme`).
.. _localizing-the-theme:
Localizing the theme
---------------------
We manage localizations on the `PyData Sphinx Theme project on Transifex <https://explore.transifex.com/12rambau/pydata-sphinx-theme/>`__.
To contribute localization, follow these steps:
#. Sign up for a `Transifex account <https://www.transifex.com/signup/>`__.
#. Join the `PyData Sphinx Theme project <https://explore.transifex.com/12rambau/pydata-sphinx-theme/>`__.
#. Select the language you want to localize. If the language you are looking for is not listed, you can `open an issue on GitHub to request it <https://github.com/pydata/pydata-sphinx-theme/issues>`__. Then you can open a pull request
to add the new language following the steps outlined in :ref:`adding-new-language`.
#. Now you are ready to start localizing the theme. If you are new to Transifex you can visit the
`Transifex documentation <https://help.transifex.com/en/articles/6240403-translating-html-content>`__ for more information.
Once you have completed your localization, the PyData Sphinx Theme maintainers will review and approve it.
Localization tips
-----------------
Localize phrases, not words
````````````````````````````
Full sentences and clauses must always be a single localizable string.
Otherwise, you can get ``next page`` localizated as ``suivant page`` instead of as ``page suivante``, etc.
Dealing with variables and markup in localizations
``````````````````````````````````````````````````
A localizable string can be a combination of a fixed string and a variable, for example, ``Welcome to the Spanish version of the site``
is a combination of the fixed parts ``Welcome to the`` and ``version of the site`` and the variable part ``Spanish``.
.. code-block:: jinja
{% trans language=language %}
Welcome to the {{ language }} version of the site
{% endtrans %}
Binding the variable as ``language=language`` ensures the string can be properly localized, especially as the word order
may vary across locales.
The above string will be extracted as ``Welcome to the %(language) version of the site``.
The translator must use ``%(language)`` verbatim while localizing the theme.
When a block contains HTML with attributes, those which don't need to be localized should be passed as arguments.
This ensures strings won't need to be re-localized if those attributes change:
.. code-block:: jinja
{% trans url="https://pydata.org/" %}
Please visit <a href="{{ url }}" title="PyData website">the PyData website</a> for more information.
{% endtrans %}
References
----------
I18N and L10N are deep topics. Here, we only cover the bare minimum needed to fulfill basic technical tasks. You might like:
- `Internationalis(z)ing Code <https://www.youtube.com/watch?v=0j74jcxSunY>`__ by Computerphile on YouTube
- `Falsehoods Programmers Believe About Language <http://garbled.benhamill.com/2017/04/18/falsehoods-programmers-believe-about-language>`__ by Ben Hamill
|
