File: speedup.rst

package info (click to toggle)
python-charset-normalizer 3.4.3-1
  • links: PTS, VCS
  • area: main
  • in suites: forky, sid
  • size: 712 kB
  • sloc: python: 5,434; makefile: 25; sh: 17
file content (50 lines) | stat: -rw-r--r-- 1,520 bytes parent folder | download | duplicates (2)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
Optional speedup extension
==========================

Why?
----

charset-normalizer will always remain pure Python, meaning that a environment without any build capabilities will
run this program without any additional requirements.

Nonetheless, starting from the version 3.0 we introduce and publish some platform specific wheels including a
pre-built extension.

Most of the time is spent in the module `md.py` so we decided to "compile it" using Mypyc.

(1) It does not require to have a separate code base
(2) Our project code base is rather simple and lightweight
(3) Mypyc is robust enough today
(4) Four times faster!

How?
----

If your platform and/or architecture is not served by this swift optimization you may compile it easily yourself.
Following those instructions (provided you have the necessary toolchain installed):

  ::

    export CHARSET_NORMALIZER_USE_MYPYC=1
    pip install mypy build wheel
    pip install charset-normalizer --no-binary :all:


How not to?
-----------

You may install charset-normalizer without the speedups by directly using the universal wheel
(most likely hosted on PyPI or any valid mirror you use) with ``--no-binary``.

E.g. when installing ``requests`` and you don't want to use the ``charset-normalizer`` speedups, you can do:

  ::

    pip install requests --no-binary charset-normalizer


When installing `charset-normalizer` by itself, you can also pass ``:all:`` as the specifier to ``--no-binary``.

  ::

    pip install charset-normalizer --no-binary :all: