1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
|
Installation
============
If you are upgrading from older versions of LinkChecker you should
also read the upgrading documentation stored in upgrading.txt.
Installing a LinkChecker release uses pre-built distribution packages. Building
the distribution packages requires hatchling_ and hatch-vcs_, and for application
translations to be compiled polib_ needs to be installed. After the sdist/wheel
has been built polib_ can be removed. pip-run_ may be useful for this.
There are several steps to resolve problems with detecting the character
encoding of checked HTML pages:
first ensure the web server, if used, is not returning an incorrect charset in
the Content-Type header; second, if possible add a meta element to the HTML
page with the correct charset; finally, check chardet_ is not installed,
Requests >= 2.26 will install charset-normalizer_, Beautiful Soup has
its own encoding detector but will use in order of preference cchardet_,
chardet_ or charset-normalizer_ (Beautiful Soup >= 4.11). You might find that
one of the other three detectors works better for your pages.
There may already be a system copy of e.g. chardet installed;
installing LinkChecker in a Python venv_ gives control over which packages are
used.
.. _chardet: https://pypi.org/project/chardet/
.. _charset-normalizer: https://pypi.org/project/charset-normalizer/
.. _pip-run: https://pypi.org/project/pip-run/
.. _cchardet: https://pypi.org/project/cchardet/
.. _polib: https://pypi.org/project/polib/
.. _hatchling: https://pypi.org/project/hatchling/
.. _hatch-vcs: https://pypi.org/project/hatch-vcs/
.. _venv: https://docs.python.org/3/library/venv.html#creating-virtual-environments
Setup with pip(x)
------------------
pip_ or pipx_ can be used to install LinkChecker on the local system.
To install the latest release from PyPI:
``pip3 install linkchecker``
or:
``pipx install linkchecker``
There is no need to wait for releases, every update to LinkChecker gets a unique
version number and is subjected to the test suite.
You can easily install the latest source from the LinkChecker GitHub repository.
First, if you want application translations:
``pip3 install polib``
Then:
``pip3 install https://github.com/linkchecker/linkchecker/archive/master.tar.gz``
.. _pip: https://pypi.org/project/pip/
.. _pipx: https://pypi.org/project/pipx/
Setup for Windows
-----------------
Python from the Microsoft Store does include pip_, but installing
within Windows Subsystem for Linux (WSL) is the preferred option:
https://docs.microsoft.com/en-us/windows/python/beginners
Setup for macOS
------------------
Python from Homebrew includes pip_. Otherwise ``python3 -m ensurepip --upgrade`` can be
used to install pip_ (untested):
https://pip.pypa.io/en/stable/installation/
Setup for GNU/Linux
-------------------
On major Linux distributions (Debian, Gentoo, Fedora, Ubuntu),
the ``linkchecker`` package is available for installation.
To install the latest LinkChecker pip_ will be available, often as a package
e.g. ``python3-pip``, a pipx_ package may also be found e.g. ``pipx``.
You may wish to install your distribution's copies of LinkChecker's dependencies
before using pip to install LinkChecker. e.g. for Debian/Ubuntu:
``apt install python3-bs4 python3-dnspython python3-requests``
If those packages are too old pip will install newer versions.
To use the dependencies from your distribution if installing LinkChecker with pipx:
``pipx install --system-site-packages linkchecker``
Manual setup for Unix systems
-----------------------------
First, install the required software.
1. Python hatchling package from https://pypi.org/project/hatchling/
2. Python hatch-vcs package from https://pypi.org/project/hatch-vcs/
3. Python Requests package from https://pypi.org/project/requests/
4. Python Beautiful Soup package from https://pypi.org/project/beautifulsoup4/
5. Python dnspython package from https://pypi.org/project/dnspython/
6. *Optional, build time only, for translations:*
Python polib package from https://pypi.org/project/polib/
7. *Optional, for bash-completion:*
Python argcomplete package from https://pypi.org/project/argcomplete/
8. *Optional, for displaying country codes:*
Python GeoIP package from https://pypi.org/project/GeoIP/
9. *Optional, for reading PDF files:*
Python pdfminer.six package from https://pypi.org/project/pdfminer.six/
10. *Optional, used for Virus checking:*
ClamAv from https://www.clamav.net/
11. *Optional, to run the WSGI web interface:*
Apache from https://httpd.apache.org/
mod_wsgi from https://pypi.org/project/mod-wsgi/
Note for developers: if you want to regenerate the po/linkchecker.pot template
from the source files, you will need xgettext with Python support. This is
available in gettext >= 0.12.
Clone the LinkChecker repository:
``git clone https://github.com/linkchecker/linkchecker.git``
``cd linkchecker``
Build the distribution wheel:
``hatchling build``
Now install the application from the wheel:
``pip install --no-index --user dist/LinkChecker-<version>-py3-none-any.whl``
Note that you may have to adjust your PATH and PYTHONPATH environment variables,
eg. by adding the commands ``export PYTHONPATH=$HOME/lib/python`` and
``export PATH=$PATH:$HOME/bin`` to your shell configuration file.
For more information look at the `Modifying Python's search path`_
documentation.
.. _Modifying Python's search path:
https://docs.python.org/3/install/#inst-search-path
After installation
------------------
LinkChecker is now installed. Have fun!
WSGI web interface
-----------------------
The included WSGI script can run LinkChecker with a nice graphical web
interface.
You can use and adjust the example HTML files in the lconline directory
to run the script.
1. Note that running LinkChecker requires CPU and memory resources.
Allowing a WSGI script to execute such a program for possibly a
large number of users might deplete those resources.
Be sure to only allow access from trusted sites to this script.
2. Copy the script lc.wsgi in the WSGI directory.
3. Adjust the "action=..." parameter in lconline/lc_cgi.html
to point to your WSGI script.
4. If you use Apache, copy config/linkchecker.apache2.conf
into your Apache configuration directory (eg. /etc/apache2/conf.d)
and enable it.
5. Load the lconline/index.html file, enter an URL and click on the
check button.
6. If something goes wrong, check the following:
a) look in the error log of your web server
b) be sure that you have enabled WSGI support in your web server,
for example by installing mod_wsgi for Apache
c) be sure that you have enabled the negotiation and versioning
modules for Apache:
a2enmod version
a2enmod negotiation
|
