1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
|
Getting the Most out of the Sitemap
===================================
Search Engine Optimization
--------------------------
Using robots.txt
^^^^^^^^^^^^^^^^
Add a **robots.txt** file in the **source** directory which has a link to the ``sitemap.xml`` or ``sitemapindex.xml`` file. For example::
User-agent: *
Sitemap: https://my-site.com/docs/sitemap.xml
Then, add **robots.txt** to :confval:`html_extra_path` in **conf.py**:
.. code-block:: python
html_extra_path = ['robots.txt']
Submit Sitemap to Search Engines
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Submit the ``sitemap.xml`` or ``sitemapindex.xml`` to the appropriate search engine tools.
Site Search Optimization
------------------------
Site search crawlers can also take advantage of sitemaps as starting points for crawling.
Examples:
- `Algolia`_
.. _Algolia: https://www.algolia.com/doc/tools/crawler/apis/configuration/sitemaps/
|
