.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples\plots\partial-dependence-plot.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_plots_partial-dependence-plot.py>`
        to download the full example code or to run this example in your browser via Binder

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_plots_partial-dependence-plot.py:


========================
Partial Dependence Plots
========================

Sigurd Carlsen Feb 2019
Holger Nahrstaedt 2020

.. currentmodule:: skopt

Plot objective now supports optional use of partial dependence as well as
different methods of defining parameter values for dependency plots.

.. GENERATED FROM PYTHON SOURCE LINES 14-23

.. code-block:: Python


    print(__doc__)
    import numpy as np

    from skopt import forest_minimize
    from skopt.plots import plot_objective

    np.random.seed(123)








.. GENERATED FROM PYTHON SOURCE LINES 24-28

Objective function
==================
Plot objective now supports optional use of partial dependence as well as
different methods of defining parameter values for dependency plots

.. GENERATED FROM PYTHON SOURCE LINES 28-38

.. code-block:: Python



    # Here we define a function that we evaluate.
    def funny_func(x):
        s = 0
        for i in range(len(x)):
            s += (x[i] * i) ** 2
        return s









.. GENERATED FROM PYTHON SOURCE LINES 39-42

Optimisation using decision trees
=================================
We run forest_minimize on the function

.. GENERATED FROM PYTHON SOURCE LINES 42-51

.. code-block:: Python

    bounds = [
        (-1, 1.0),
    ] * 3
    n_calls = 50

    result = forest_minimize(
        funny_func, bounds, n_calls=n_calls, base_estimator="ET", random_state=4
    )








.. GENERATED FROM PYTHON SOURCE LINES 52-58

Partial dependence plot
=======================
Here we see an example of using partial dependence. Even when setting
n_points all the way down to 10 from the default of 40, this method is
still very slow. This is because partial dependence calculates 250 extra
predictions for each point on the plots.

.. GENERATED FROM PYTHON SOURCE LINES 58-62

.. code-block:: Python



    _ = plot_objective(result, n_points=10)




.. image-sg:: /auto_examples/plots/images/sphx_glr_partial-dependence-plot_001.png
   :alt: partial dependence plot
   :srcset: /auto_examples/plots/images/sphx_glr_partial-dependence-plot_001.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 63-67

It is possible to change the location of the red dot, which normally shows
the position of the found minimum. We can set it 'expected_minimum',
which is the minimum value of the surrogate function, obtained by a
minimum search method.

.. GENERATED FROM PYTHON SOURCE LINES 67-69

.. code-block:: Python


    _ = plot_objective(result, n_points=10, minimum='expected_minimum')



.. image-sg:: /auto_examples/plots/images/sphx_glr_partial-dependence-plot_002.png
   :alt: partial dependence plot
   :srcset: /auto_examples/plots/images/sphx_glr_partial-dependence-plot_002.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 70-76

Plot without partial dependence
===============================
Here we plot without partial dependence. We see that it is a lot faster.
Also the values for the other parameters are set to the default "result"
which is the parameter set of the best observed value so far. In the case
of funny_func this is close to 0 for all parameters.

.. GENERATED FROM PYTHON SOURCE LINES 76-79

.. code-block:: Python


    _ = plot_objective(result, sample_source='result', n_points=10)




.. image-sg:: /auto_examples/plots/images/sphx_glr_partial-dependence-plot_003.png
   :alt: partial dependence plot
   :srcset: /auto_examples/plots/images/sphx_glr_partial-dependence-plot_003.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 80-86

Modify the shown minimum
========================
Here we try with setting the `minimum` parameters to something other than
"result". First we try with "expected_minimum" which is the set of
parameters that gives the miniumum value of the surrogate function,
using scipys minimum search method.

.. GENERATED FROM PYTHON SOURCE LINES 86-91

.. code-block:: Python


    _ = plot_objective(
        result, n_points=10, sample_source='expected_minimum', minimum='expected_minimum'
    )




.. image-sg:: /auto_examples/plots/images/sphx_glr_partial-dependence-plot_004.png
   :alt: partial dependence plot
   :srcset: /auto_examples/plots/images/sphx_glr_partial-dependence-plot_004.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 92-94

"expected_minimum_random" is a naive way of finding the minimum of the
surrogate by only using random sampling:

.. GENERATED FROM PYTHON SOURCE LINES 94-102

.. code-block:: Python


    _ = plot_objective(
        result,
        n_points=10,
        sample_source='expected_minimum_random',
        minimum='expected_minimum_random',
    )




.. image-sg:: /auto_examples/plots/images/sphx_glr_partial-dependence-plot_005.png
   :alt: partial dependence plot
   :srcset: /auto_examples/plots/images/sphx_glr_partial-dependence-plot_005.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 103-106

We can also specify how many initial samples are used for the two different
"expected_minimum" methods. We set it to a low value in the next examples
to showcase how it affects the minimum for the two methods.

.. GENERATED FROM PYTHON SOURCE LINES 106-115

.. code-block:: Python


    _ = plot_objective(
        result,
        n_points=10,
        sample_source='expected_minimum_random',
        minimum='expected_minimum_random',
        n_minimum_search=10,
    )




.. image-sg:: /auto_examples/plots/images/sphx_glr_partial-dependence-plot_006.png
   :alt: partial dependence plot
   :srcset: /auto_examples/plots/images/sphx_glr_partial-dependence-plot_006.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 116-125

.. code-block:: Python


    _ = plot_objective(
        result,
        n_points=10,
        sample_source="expected_minimum",
        minimum='expected_minimum',
        n_minimum_search=2,
    )




.. image-sg:: /auto_examples/plots/images/sphx_glr_partial-dependence-plot_007.png
   :alt: partial dependence plot
   :srcset: /auto_examples/plots/images/sphx_glr_partial-dependence-plot_007.png
   :class: sphx-glr-single-img





.. GENERATED FROM PYTHON SOURCE LINES 126-130

Set a minimum location
======================
Lastly we can also define these parameters ourself by parsing a list
as the minimum argument:

.. GENERATED FROM PYTHON SOURCE LINES 130-134

.. code-block:: Python


    _ = plot_objective(
        result, n_points=10, sample_source=[1, -0.5, 0.5], minimum=[1, -0.5, 0.5]
    )



.. image-sg:: /auto_examples/plots/images/sphx_glr_partial-dependence-plot_008.png
   :alt: partial dependence plot
   :srcset: /auto_examples/plots/images/sphx_glr_partial-dependence-plot_008.png
   :class: sphx-glr-single-img






.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 14.903 seconds)


.. _sphx_glr_download_auto_examples_plots_partial-dependence-plot.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: binder-badge

      .. image:: images/binder_badge_logo.svg
        :target: https://mybinder.org/v2/gh/holgern/scikit-optimize/master?urlpath=lab/tree/notebooks/auto_examples/plots/partial-dependence-plot.ipynb
        :alt: Launch binder
        :width: 150 px

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: partial-dependence-plot.ipynb <partial-dependence-plot.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: partial-dependence-plot.py <partial-dependence-plot.py>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_