Proxies and Wrappers
====================

Underlying the implementation of the decorators created by the **wrapt**
module is a wrapper class which acts as a transparent object proxy. This
document describes the object proxy and the various custom wrappers provided.

Object Proxy
------------

The object proxy class is available as ``wrapt.ObjectProxy``. The class
would not normally be used directly, but as a base class to custom object
proxies or wrappers which add behaviour which overrides that of the
original object. When an object proxy is used, it will pass through any
actions performed on the proxy through to the wrapped object.

::

    >>> table = {}
    >>> proxy = wrapt.ObjectProxy(table)
    >>> proxy['key-1'] = 'value-1'
    >>> proxy['key-2'] = 'value-2'

    >>> proxy.keys()
    ['key-2', 'key-1']
    >>> table.keys()
    ['key-2', 'key-1']

    >>> isinstance(proxy, dict)
    True

    >>> dir(proxy)
    ['__class__', '__cmp__', '__contains__', '__delattr__', '__delitem__',
    '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__',
    '__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__',
    '__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__',
    '__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__',
    '__subclasshook__', 'clear', 'copy', 'fromkeys', 'get', 'has_key',
    'items', 'iteritems', 'iterkeys', 'itervalues', 'keys', 'pop',
    'popitem', 'setdefault', 'update', 'values']

This ability for a proxy to stand in for the original goes as far as
arithmetic operations, rich comparison and hashing.

::

    >>> value = 1
    >>> proxy = wrapt.ObjectProxy(value)

    >>> proxy + 1
    2

    >>> int(proxy)
    1
    >>> hash(proxy)
    1
    >>> hash(value)
    1

    >>> proxy < 2
    True
    >>> proxy == 0
    False

Do note however, that when wrapping an object proxy around a literal value,
the original value is effectively copied into the proxy object and any
operation which updates the value will only update the value held by the
proxy object.

::

    >>> value = 1
    >>> proxy = wrapt.ObjectProxy(value)
    >>> type(proxy)
    <type 'ObjectProxy'>

    >>> proxy += 1

    >>> type(proxy)
    <type 'ObjectProxy'>

    >>> print(proxy)
    2
    >>> print(value)
    1

Object wrappers may therefore have limited use in conjunction with literal
values.

Type Comparison
---------------

The type of an instance of the object proxy will be ``ObjectProxy``, or that
of any derived class type if creating a custom object proxy.

::

    >>> value = 1
    >>> proxy = wrapt.ObjectProxy(value)
    >>> type(proxy)
    <type 'ObjectProxy'>

    >>> class CustomProxy(wrapt.ObjectProxy):
    ...     pass

    >>> proxy = CustomProxy(1)

    >>> type(proxy)
    <class '__main__.CustomProxy'>

Direct type comparisons in Python are generally frowned upon and allowance
for 'duck typing' preferred. Instead of direct type comparison, the
``isinstance()`` function would therefore be used. Using ``isinstance()``,
comparison of the type of the object proxy will properly evaluate against
the wrapped object.

::

    >>> isinstance(proxy, int)
    True

This works because the ``__class__`` attribute actually returns the class
type for the wrapped object.

::

    >>> proxy.__class__
    <type 'int'>

Note that ``isinstance()`` will still also succeed if comparing to the
``ObjectProxy`` type. It is therefore still possible to use ``isinstance()``
to determine if an object is an object proxy.

::

    >>> isinstance(proxy, wrapt.ObjectProxy)
    True

    >>> class CustomProxy(wrapt.ObjectProxy):
    ...     pass

    >>> proxy = CustomProxy(1)

    >>> isinstance(proxy, wrapt.ObjectProxy)
    True
    >>> isinstance(proxy, CustomProxy)
    True


Custom Object Proxies
---------------------

A custom proxy is where one creates a derived object proxy and overrides
some specific behaviour of the proxy.

::

    def function():
        print('executing', function.__name__)

    class CallableWrapper(wrapt.ObjectProxy):

        def __call__(self, *args, **kwargs):
            print('entering', self.__wrapped__.__name__)
            try:
                return self.__wrapped__(*args, **kwargs)
            finally:
                print('exiting', self.__wrapped__.__name__)

    >>> proxy = CallableWrapper(function)

    >>> proxy()
    ('entering', 'function')
    ('executing', 'function')
    ('exiting', 'function')

Any method of the original wrapped object can be overridden, including
special Python methods such as ``__call__()``. If it is necessary to change
what happens when a specific attribute of the wrapped object is accessed,
then properties can be used.

If it is necessary to access the original wrapped object from within an
overridden method or property, then ``self.__wrapped__`` is used.

Proxy Object Attributes
-----------------------

When an attempt is made to access an attribute from the proxy, the same
named attribute would in normal circumstances be accessed from the wrapped
object. When updating an attributes value, or deleting the attribute, that
change will also be reflected in the wrapped object.

::

    >>> proxy = CallableWrapper(function)

    >>> hasattr(function, 'attribute')
    False
    >>> hasattr(proxy, 'attribute')
    False

    >>> proxy.attribute = 1

    >>> hasattr(function, 'attribute')
    True
    >>> hasattr(proxy, 'attribute')
    True

    >>> function.attribute
    1
    >>> proxy.attribute
    1

If an attribute was updated on the wrapped object directly, that change is
still reflected in what is available via the proxy.

::

    >>> function.attribute = 2

    >>> function.attribute
    2
    >>> proxy.attribute
    2

If creating a custom proxy and it needs to keep attributes of its own which
should not be saved through to the wrapped object, those attributes should
be prefixed with ``_self_``.

::

    def function():
        print('executing', function.__name__)

    class CallableWrapper(wrapt.ObjectProxy):

        def __init__(self, wrapped, wrapper):
            super(CallableWrapper, self).__init__(wrapped)
            self._self_wrapper = wrapper

        def __call__(self, *args, **kwargs):
            return self._self_wrapper(self.__wrapped__, args, kwargs)

    def wrapper(wrapped, args, kwargs):
          print('entering', wrapped.__name__)
          try:
              return wrapped(*args, **kwargs)
          finally:
              print('exiting', wrapped.__name__)

    >>> proxy = CallableWrapper(function, wrapper)

    >>> proxy._self_wrapper
    <function wrapper at 0x1005961b8>

    >>> function._self_wrapper
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'function' object has no attribute '_self_wrapper'

If an attribute local to the proxy must be available under a name without
this special prefix, then a ``@property`` can be used in the class
definition.

::

    class CustomProxy(wrapt.ObjectProxy):

        def __init__(self, wrapped):
            super(CustomProxy, self).__init__(wrapped)
            self._self_attribute = 1

        @property
        def attribute(self):
            return self._self_attribute

        @attribute.setter
        def attribute(self, value):
            self._self_attribute = value

        @attribute.deleter
        def attribute(self):
           del self._self_attribute

    >>> proxy = CustomProxy(1)
    >>> print proxy.attribute
    1
    >>> proxy.attribute = 2
    >>> print proxy.attribute
    2
    >>> del proxy.attribute
    >>> print proxy.attribute
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'int' object has no attribute 'attribute'

Alternatively, the attribute can be specified as a class attribute, with
that then being overridden if necessary, with a specific value in the
``__init__()`` method of the class.

::

    class CustomProxy(wrapt.ObjectProxy):
        attribute = None
        def __init__(self, wrapped):
            super(CustomProxy, self).__init__(wrapped)
            self.attribute = 1

    >>> proxy = CustomProxy(1)
    >>> print proxy.attribute
    1
    >>> proxy.attribute = 2
    >>> print proxy.attribute
    2
    >>> del proxy.attribute
    >>> print proxy.attribute
    None

Just be aware that although the attribute can be deleted from the instance
of the custom proxy, lookup will then fallback to using the class attribute.

Special Object Methods
----------------------

The ``ObjectProxy`` class implements most of the special builtin methods of a
Python object, such as ``__len__()``, ``__getitem__()``, ``__setitem__()``,
``__delitem__()`` etc. This allows the proxy to be used in place of the
original object with operations on the proxy being passed through to the
wrapped object as appropriate.

Some special methods are not implemented by the ``ObjectProxy`` class by default
because their presence could affect the original code which interacted with the
wrapped object. Examples of methods which are excluded are ``__get__()``,
``__set__()`` and ``__delete__()``, as well as ``__call__()``, iterator methods
and awaitable methods. If it is necessary for a custom proxy to implement
one of these special methods, then it can be done by overriding the method in
a derived custom object proxy class.

That said, note that due to a bad design decision in the original ``ObjectProxy``
class, the ``__iter__()`` method was implemented when it should not have been.
That this should not have been was only realized some time after the original
release. The presence of ``__iter__()`` means that the proxy will always
appear to be iterable, even when the wrapped object is not. This can lead to
confusion and bugs. If the wrapped object is not iterable, then attempting to
iterate over the proxy will result in an exception, but if code checks for
iterability using ``hasattr(proxy, '__iter__')`` then that will always return
``True``. The presence of ``__iter__()`` in ``ObjectProxy`` is therefore
considered a bug, but one which cannot be fixed without breaking backwards
compatibility.

If the presence of ``__iter__()`` is causing problems, rather than deriving
from ``ObjectProxy``, from **wrapt** version 2.0.0, it is possible to create a
custom proxy by deriving directly from ``wrapt.BaseObjectProxy`` which does not
implement ``__iter__()``. To maintain backward compatibility, ``ObjectProxy``
class derives from ``BaseObjectProxy`` and adds the existing ``__iter__()``
method implementation.

If you require the use of **wrapt** version 2.0.0 or later for any reasons,
it is actually recommended to completely avoid using ``ObjectProxy`` and instead
always derive from ``BaseObjectProxy``. Doing this will though mean your code
will not work with prior versions of **wrapt**.

If for some reason you feel needing to manually add the excluded special
methods in a custom object proxy is annoying, you can instead use
``wrapt.AutoObjectProxy`` as the base class. This class will automatically add
the special methods which are excluded by default. Be aware though that this
will result in a new class type being dynamically created on the fly for each
instantiation of the custom proxy. This will result in the memory requirement
for each object proxy instance being higher than normal. The ``AutoObjectProxy``
class should therefore only be used when absolutely necessary and never in
situations where a large number of proxy instances are being created.

Function Wrappers
-----------------

Although an ``ObjectProxy`` can be used to wrap a function, it doesn't do
anything special in respect of bound methods. If attempting to use a custom
object proxy to wrap instance methods, class methods or static methods, it
would be necessary to override the appropriate descriptor protocol methods
in order to be able to intercept and modify in any way the execution of the
wrapped function.

::

    class BoundCallableWrapper(wrapt.ObjectProxy):

        def __init__(self, wrapped, wrapper):
            super(BoundCallableWrapper, self).__init__(wrapped)
            self._self_wrapper = wrapper

        def __get__(self, instance, owner):
            return self

        def __call__(self, *args, **kwargs):
            return self._self_wrapper(self.__wrapped__, args, kwargs)

    class CallableWrapper(wrapt.ObjectProxy):

        def __init__(self, wrapped, wrapper):
            super(CallableWrapper, self).__init__(wrapped)
            self._self_wrapper = wrapper

        def __get__(self, instance, owner):
            function = self.__wrapped__.__get__(instance, owner)
            return BoundCallableWrapper(function, self._self_wrapper)

        def __call__(self, *args, **kwargs):
            return self._self_wrapper(self.__wrapped__, args, kwargs)

The ``CallableWrapper.__call__()`` method would therefore be invoked when
``CallableWrapper`` is used around a regular function. The
``BoundCallableWrapper.__call__()`` would instead be what is invoked for a
bound method, the instance of ``BoundCallableWrapper`` having being created
when the original wrapped method was bound to the class instance.

This specific pattern is actually the basis of what is required to
implement a robust function wrapper for use in implementing a decorator.
Because it is a fundamental pattern, a predefined version is available as
``wrapt.FunctionWrapper``.

As with the illustrative example above, ``FunctionWrapper`` class accepts
two key arguments:

* ``wrapped`` - The function being wrapped.
* ``wrapper`` - A wrapper function to be called when the wrapped function is invoked.

Although in prior examples the wrapper function was shown as accepting three
positional arguments of the wrapped function and the args and kwargs for when
the wrapped function was called, when using ``FunctionWrapper``, it is
expected that the wrapper function accepts four arguments. These are:

* ``wrapped`` - The wrapped function which in turns needs to be called by your wrapper function.
* ``instance`` - The object to which the wrapped function was bound when it was called.
* ``args`` - The list of positional arguments supplied when the decorated function was called.
* ``kwargs`` - The dictionary of keyword arguments supplied when the decorated function was called.

When ``FunctionWrapper`` is applied to a normal function or static method,
the wrapper function when called will be passed ``None`` as the
``instance`` argument.

When applied to an instance method, the wrapper function when called will
be passed the instance of the class the method is being called on as the
``instance`` argument. This will be the case even when the instance method
was called explicitly via the class and the instance passed as the first
argument. That is, the instance will never be passed as part of ``args``.

When applied to a class method, the wrapper function when called will be
passed the class type as the ``instance`` argument.

When applied to a class, the wrapper function when called will be passed
``None`` as the ``instance`` argument. The ``wrapped`` argument in this
case will be the class.

The above rules can be summarised with the following example.

::

    import inspect

    def wrapper(wrapped, instance, args, kwargs):
        if instance is None:
            if inspect.isclass(wrapped):
                # Decorator was applied to a class.
                return wrapped(*args, **kwargs)
            else:
                # Decorator was applied to a function or staticmethod.
                return wrapped(*args, **kwargs)
        else:
            if inspect.isclass(instance):
                # Decorator was applied to a classmethod.
                return wrapped(*args, **kwargs)
            else:
                # Decorator was applied to an instancemethod.
                return wrapped(*args, **kwargs)

Using these checks it is therefore possible to create a universal function
wrapper that can be applied in all situations. It is no longer necessary to
create different variants of function wrappers for normal functions and
instance methods.

In all cases, the wrapped function passed to the wrapper function is called
in the same way, with ``args`` and ``kwargs`` being passed. The
``instance`` argument doesn't need to be used in calling the wrapped
function.

A simple decorator factory implementation which makes use of
``FunctionWrapper`` to delegate execution of the wrapped function to
the wrapper function  would be:

::

    def function_wrapper(wrapper):
        @functools.wraps(wrapper)
        def _wrapper(wrapped):
            return FunctionWrapper(wrapped, wrapper)
        return _wrapper

It would be used like:

::

    @function_wrapper
    def wrapper(wrapped, instance, args, kwargs):
        return wrapped(*args, **kwargs)

    @wrapper
    def function():
        pass

This example of a simplified decorator factory is made available as
``wrapt.function_wrapper``. Although it is usable in its own right, it is
preferable that ``wrapt.decorator`` be used to create decorators as it
provides additional features. The ``@function_wrapper`` decorator would
generally be used more when performing monkey patching and needing to
dynamically create function wrappers.

::

    @function_wrapper
    def wrapper(wrapped, instance, args, kwargs):
        return wrapped(*args, **kwargs)

    callback = wrapper(fetch_callback())

Custom Function Wrappers
------------------------

If it is necessary to implement a custom function wrapper in order to
override the behaviour of a wrapped function, it is possible to still
derive from the ``wrapt.FunctionWrapper`` class. That binding of functions
can occur does however complicate things. This is because the bound
function is a separate object implemented as a different type.

The type of the separate bound function wrapper is
``wrapt.BoundFunctionWrapper``. If the behaviour for the bound function
also needs to be overridden, a derived version of this class will also
need to be created. The derived custom function wrapper will then need
to indicate that this second type should be used when creating the bound
function wrapper, rather than the default. This is done via the
``__bound_function_wrapper__`` attribute of the class.

::

    def custom_function_wrapper(attribute):

        class CustomBoundFunctionWrapper(wrapt.BoundFunctionWrapper):

            def __call__(self, *args, **kwargs):
                if attribute:
                    ...
                return super(CustomBoundFunctionWrapper, self).__call__(*args, **kwargs)

        class CustomFunctionWrapper(wrapt.FunctionWrapper):

            __bound_function_wrapper__ = CustomBoundFunctionWrapper

            def __call__(self, *args, **kwargs):
                if attribute:
                    ...
                return super(CustomFunctionWrapper, self).__call__(*args, **kwargs)

        return CustomFunctionWrapper

Note that to preserve the existing convention as to what arguments are
accepted by the constructors of both ``wrapt.FunctionWrapper`` and
``wrapt.BoundFunctionWrapper`` a function closure is used in this example,
with the classes defined within the closure. The benefit of this approach
is that the custom function wrapper can then be used with
``@wrapt.decorator``, with the default use of ``FunctionWrapper`` being
replaced with the custom function wrapper.

::

    @wrapt.decorator(proxy=custom_function_wrapper("attribute"))
    def wrapper(wrapped, instance, args, kwargs):
        return wrapped(*args, **kwargs)

If it is necessary to set up instance variables on the function wrappers
because the value needs to change over the lifetime of that instance of
the function wrapper, constructors can be defined to add the attributes on
the instance, but these should just pass all positional and keyword
parameters as is through to the base class.

::

    def custom_function_wrapper(attribute):

        class CustomBoundFunctionWrapper(wrapt.BoundFunctionWrapper):

            def __init__(self, *args, **kwargs):
                super(CustomBoundFunctionWrapper, self).__init(*args, **kwargs)
                self._self_attribute = attribute

            def __call__(self, *args, **kwargs):
                if self._self_attribute:
                    ...
                return super(CustomBoundFunctionWrapper, self).__call__(*args, **kwargs)

        class CustomFunctionWrapper(wrapt.FunctionWrapper):

            __bound_function_wrapper__ = CustomBoundFunctionWrapper

            def __init__(self, *args, **kwargs):
                super(CustomFunctionWrapper, self).__init(*args, **kwargs)
                self._self_attribute = attribute

            def __call__(self, *args, **kwargs):
                if self._self_attribute:
                    ...
                return super(CustomFunctionWrapper, self).__call__(*args, **kwargs)

        return CustomFunctionWrapper

If the bound function wrapper needs to be able to access back to the parent
function wrapper it was created from, it can use ``self._self_parent``.

::

    def custom_function_wrapper(attribute):

        class CustomBoundFunctionWrapper(wrapt.BoundFunctionWrapper):

            def __call__(self, *args, **kwargs):
                if self._self_parent._self_attribute:
                    ...
                return super(CustomBoundFunctionWrapper, self).__call__(*args, **kwargs)

        class CustomFunctionWrapper(wrapt.FunctionWrapper):

            __bound_function_wrapper__ = CustomBoundFunctionWrapper

            def __init__(self, *args, **kwargs):
                super(CustomFunctionWrapper, self).__init(*args, **kwargs)
                self._self_attribute = attribute

            def __call__(self, *args, **kwargs):
                if self._self_attribute:
                    ...
                return super(CustomFunctionWrapper, self).__call__(*args, **kwargs)

        return CustomFunctionWrapper

Lazy Object Proxies
-------------------

The ``ObjectProxy`` and ``BaseObjectProxy`` classes require that the
wrapped object be supplied at the time the proxy is created. In some
situations this may not be possible or desirable. For example, if the
wrapped object is expensive to create and may not be needed, or if the
wrapped object is not available at the time the proxy must be created.

To address this, the ``wrapt.LazyObjectProxy`` class is provided. This class
derives from ``AutoObjectProxy`` and allows the wrapped object to be
supplied at a later time via a callable which will only be called the first
time the wrapped object is needed.

The callable which is used to create the wrapped object is supplied as
the ``factory`` argument to the constructor of ``LazyObjectProxy``. The
``factory`` callable should accept no arguments and return the object to
be wrapped.

An example of using ``LazyObjectProxy`` is to lazily import a Python module,
with the module only being imported when it is first needed. This can be
done by using the built-in ``__import__()`` function within a factory function.
One can even optionally specify an attribute of the module to be retrieved
and used as the wrapped object instead of the module itself.

::

    def lazy_import(name, attribute=None, *, interface=...):
        """Lazily imports the module `name`, returning a `LazyObjectProxy` which
        will import the module when it is first needed. When `name is a dotted name,
        then the full dotted name is imported and the last module is taken as the
        target. If `attribute` is provided then it is used to retrieve an attribute
        from the module.
        """

        if attribute is not None:
            if interface is ...:
                interface = Callable
        else:
            if interface is ...:
                interface = ModuleType

        def _import():
            module = __import__(name, fromlist=[""])

            if attribute is not None:
                return getattr(module, attribute)

            return module

        return LazyObjectProxy(_import, interface=interface)

The ``interface`` argument is a hint as to the type of object being wrapped.
This is used to help ``LazyObjectProxy`` determine what special methods need to
be added to the proxy in order to properly stand in for the wrapped object
prior to the module being imported, at which point the required special methods
can be determined from the actual wrapped object. This is important for cases
such as where the wrapped object is a callable, as the proxy will need to
implement ``__call__()`` so that calling it will trigger the import of the module
and then call the wrapped callable. If dealing with attribute of a module which
have a different interface type, such as an iterable, then the appropriate type
from ``collections.abc`` should be used.

Since such a lazy import feature is generally useful, a convenience function
``wrapt.lazy_import()`` is provided which implements the above example.

This lazy import feature can be used to avoid the overhead of importing
modules which may not be needed, or possibly to avoid circular import problems.
It can be used in place of standard ``import`` as follows:

::

    import wrapt

    @wrapt.when_imported("graphlib")
    def module_imported(module):
        print(f"{module.__name__} imported")

    # Replaces "import graphlib".

    graphlib = wrapt.lazy_import("graphlib")

    print("waiting for import")

    print(graphlib.TopologicalSorter)

The ``lazy_import()`` function can be seen as an alternative to
`PEP 810 - Explicit lazy imports <https://peps.python.org/pep-0810/>`_ which
proposes a new syntax in Python for lazy imports. The benefit of using
``wrapt.lazy_import()`` is that it works with all current versions of Python
and does not require any changes to the Python language. As such you could
start using it today.

As ``LazyObjectProxy`` is derived from ``AutoObjectProxy``, as already mentioned
the memory requirement for each instance of ``LazyObjectProxy`` will be higher
than that of a normal ``ObjectProxy``. ``LazyObjectProxy`` should therefore only
be used when absolutely necessary and never in situations where a large number
of proxy instances are being created.