Useful Python Utils
==============================================================================

Python Utils is a collection of small Python functions and
classes which make common patterns shorter and easier. It is by no means a
complete collection but it has served me quite a bit in the past and I will
keep extending it.

One of the libraries using Python Utils is Django Utils.

Documentation is available at: https://python-utils.readthedocs.org/en/latest/

Links
-----

 - The source: https://github.com/WoLpH/python-utils
 - Project page: https://pypi.python.org/pypi/python-utils
 - Reporting bugs: https://github.com/WoLpH/python-utils/issues
 - Documentation: https://python-utils.readthedocs.io/en/latest/
 - My blog: https://wol.ph/

Security contact information
------------------------------------------------------------------------------

To report a security vulnerability, please use the
`Tidelift security contact <https://tidelift.com/security>`_.
Tidelift will coordinate the fix and disclosure.

Requirements for installing:
------------------------------------------------------------------------------

For the Python 3+ release (i.e. v3.0.0 or higher) there are no requirements.
For the Python 2 compatible version (v2.x.x) the `six` package is needed.

Installation:
------------------------------------------------------------------------------

The package can be installed through `pip` (this is the recommended method):

.. code-block:: bash

    pip install python-utils
    
Or if `pip` is not available, `easy_install` should work as well:

.. code-block:: bash

    easy_install python-utils
    
Or download the latest release from Pypi (https://pypi.python.org/pypi/python-utils) or Github.

Note that the releases on Pypi are signed with my GPG key (https://pgp.mit.edu/pks/lookup?op=vindex&search=0xE81444E9CE1F695D) and can be checked using GPG:

.. code-block:: bash

     gpg --verify python-utils-<version>.tar.gz.asc python-utils-<version>.tar.gz

Quickstart
------------------------------------------------------------------------------

This module makes it easy to execute common tasks in Python scripts such as
converting text to numbers and making sure a string is in unicode or bytes
format.

Examples
------------------------------------------------------------------------------

Automatically converting a generator to a list, dict or other collections
using a decorator:

.. code-block:: pycon

    >>> @decorators.listify()
    ... def generate_list():
    ...     yield 1
    ...     yield 2
    ...     yield 3
    ...
    >>> generate_list()
    [1, 2, 3]

    >>> @listify(collection=dict)
    ... def dict_generator():
    ...     yield 'a', 1
    ...     yield 'b', 2

    >>> dict_generator()
    {'a': 1, 'b': 2}

Retrying until timeout
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To easily retry a block of code with a configurable timeout, you can use the
`time.timeout_generator`:

.. code-block:: pycon

    >>> for i in time.timeout_generator(10):
    ...     try:
    ...         # Run your code here
    ...     except Exception as e:
    ...         # Handle the exception

Formatting of timestamps, dates and times
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Easy formatting of timestamps and calculating the time since:

.. code-block:: pycon

    >>> time.format_time('1')
    '0:00:01'
    >>> time.format_time(1.234)
    '0:00:01'
    >>> time.format_time(1)
    '0:00:01'
    >>> time.format_time(datetime.datetime(2000, 1, 2, 3, 4, 5, 6))
    '2000-01-02 03:04:05'
    >>> time.format_time(datetime.date(2000, 1, 2))
    '2000-01-02'
    >>> time.format_time(datetime.timedelta(seconds=3661))
    '1:01:01'
    >>> time.format_time(None)
    '--:--:--'

    >>> formatters.timesince(now)
    'just now'
    >>> formatters.timesince(now - datetime.timedelta(seconds=1))
    '1 second ago'
    >>> formatters.timesince(now - datetime.timedelta(seconds=2))
    '2 seconds ago'
    >>> formatters.timesince(now - datetime.timedelta(seconds=60))
    '1 minute ago'

Converting your test from camel-case to underscores:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: pycon

    >>> camel_to_underscore('SpamEggsAndBacon')
    'spam_eggs_and_bacon'

Attribute setting decorator. Very useful for the Django admin
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A convenient decorator to set function attributes using a decorator:

.. code-block:: pycon

    You can use:
    >>> @decorators.set_attributes(short_description='Name')
    ... def upper_case_name(self, obj):
    ...     return ("%s %s" % (obj.first_name, obj.last_name)).upper()

    Instead of:
    >>> def upper_case_name(obj):
    ...     return ("%s %s" % (obj.first_name, obj.last_name)).upper()

    >>> upper_case_name.short_description = 'Name'

This can be very useful for the Django admin as it allows you to have all
metadata in one place.

Scaling numbers between ranges
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: pycon

    >>> converters.remap(500, old_min=0, old_max=1000, new_min=0, new_max=100)
    50

    # Or with decimals:
    >>> remap(decimal.Decimal('250.0'), 0.0, 1000.0, 0.0, 100.0)
    Decimal('25.0')

Get the screen/window/terminal size in characters:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: pycon

    >>> terminal.get_terminal_size()
    (80, 24)

That method supports IPython and Jupyter as well as regular shells, using
`blessings` and other modules depending on what is available.

Extracting numbers from nearly every string:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: pycon

    >>> converters.to_int('spam15eggs')
    15
    >>> converters.to_int('spam')
    0
    >>> number = converters.to_int('spam', default=1)
    1

Doing a global import of all the modules in a package programmatically:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To do a global import programmatically you can use the `import_global`
function. This effectively emulates a `from ... import *`

.. code-block:: python

    from python_utils.import_ import import_global

    # The following is  the equivalent of `from some_module import *`
    import_global('some_module')

Automatically named logger for classes:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Or add a correclty named logger to your classes which can be easily accessed:

.. code-block:: python

    class MyClass(Logged):
        def __init__(self):
            Logged.__init__(self)

    my_class = MyClass()

    # Accessing the logging method:
    my_class.error('error')

    # With formatting:
    my_class.error('The logger supports %(formatting)s',
                   formatting='named parameters')

    # Or to access the actual log function (overwriting the log formatting can
    # be done n the log method)
    import logging
    my_class.log(logging.ERROR, 'log')

Alternatively loguru is also supported. It is largely a drop-in replacement for the logging module which is a bit more convenient to configure:

First install the extra loguru package:

.. code-block:: bash

    pip install 'python-utils[loguru]'

.. code-block:: python

    class MyClass(Logurud):
        ...

Now you can use the `Logurud` class to make functions such as `self.info()`
available. The benefit of this approach is that you can add extra context or
options to you specific loguru instance (i.e. `self.logger`):

Convenient type aliases and some commonly used types:

.. code-block:: python

    # For type hinting scopes such as locals/globals/vars
    Scope = Dict[str, Any]
    OptionalScope = O[Scope]

    # Note that Number is only useful for extra clarity since float
    # will work for both int and float in practice.
    Number = U[int, float]
    DecimalNumber = U[Number, decimal.Decimal]

    # To accept an exception or list of exceptions
    ExceptionType = Type[Exception]
    ExceptionsType = U[Tuple[ExceptionType, ...], ExceptionType]

    # Matching string/bytes types:
    StringTypes = U[str, bytes]