.. _whatsnew_100:

What's new in 1.0.0 (January 29, 2020)
--------------------------------------

These are the changes in pandas 1.0.0. See :ref:`release` for a full changelog
including other versions of pandas.

.. note::

    The pandas 1.0 release removed a lot of functionality that was deprecated
    in previous releases (see :ref:`below <whatsnew_100.prior_deprecations>`
    for an overview). It is recommended to first upgrade to pandas 0.25 and to
    ensure your code is working without warnings, before upgrading to pandas
    1.0.


New deprecation policy
~~~~~~~~~~~~~~~~~~~~~~

Starting with pandas 1.0.0, pandas will adopt a variant of `SemVer`_ to
version releases. Briefly,

* Deprecations will be introduced in minor releases (e.g. 1.1.0, 1.2.0, 2.1.0, ...)
* Deprecations will be enforced in major releases (e.g. 1.0.0, 2.0.0, 3.0.0, ...)
* API-breaking changes will be made only in major releases (except for experimental features)

See :ref:`policies.version` for more.

.. _2019 Pandas User Survey: https://pandas.pydata.org/community/blog/2019-user-survey.html
.. _SemVer: https://semver.org

{{ header }}

.. ---------------------------------------------------------------------------

Enhancements
~~~~~~~~~~~~

.. _whatsnew_100.numba_rolling_apply:

Using Numba in ``rolling.apply`` and ``expanding.apply``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

We've added an ``engine`` keyword to :meth:`~core.window.rolling.Rolling.apply` and :meth:`~core.window.expanding.Expanding.apply`
that allows the user to execute the routine using `Numba <https://numba.pydata.org/>`__ instead of Cython.
Using the Numba engine can yield significant performance gains if the apply function can operate on numpy arrays and
the data set is larger (1 million rows or greater). For more details, see
:ref:`rolling apply documentation <window.numba_engine>` (:issue:`28987`, :issue:`30936`)

.. _whatsnew_100.custom_window:

Defining custom windows for rolling operations
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

We've added a :func:`pandas.api.indexers.BaseIndexer` class that allows users to define how
window bounds are created during ``rolling`` operations. Users can define their own ``get_window_bounds``
method on a :func:`pandas.api.indexers.BaseIndexer` subclass that will generate the start and end
indices used for each window during the rolling aggregation. For more details and example usage, see
the :ref:`custom window rolling documentation <window.custom_rolling_window>`

.. _whatsnew_100.to_markdown:

Converting to markdown
^^^^^^^^^^^^^^^^^^^^^^

We've added :meth:`~DataFrame.to_markdown` for creating a markdown table (:issue:`11052`)

.. ipython:: python

   df = pd.DataFrame({"A": [1, 2, 3], "B": [1, 2, 3]}, index=['a', 'a', 'b'])
   print(df.to_markdown())

Experimental new features
~~~~~~~~~~~~~~~~~~~~~~~~~

.. _whatsnew_100.NA:

Experimental ``NA`` scalar to denote missing values
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

A new ``pd.NA`` value (singleton) is introduced to represent scalar missing
values. Up to now, pandas used several values to represent missing data: ``np.nan`` is used for this for float data, ``np.nan`` or
``None`` for object-dtype data and ``pd.NaT`` for datetime-like data. The
goal of ``pd.NA`` is to provide a "missing" indicator that can be used
consistently across data types. ``pd.NA`` is currently used by the nullable integer and boolean
data types and the new string data type (:issue:`28095`).

.. warning::

   Experimental: the behaviour of ``pd.NA`` can still change without warning.

For example, creating a Series using the nullable integer dtype:

.. ipython:: python

    s = pd.Series([1, 2, None], dtype="Int64")
    s
    s[2]

Compared to ``np.nan``, ``pd.NA`` behaves differently in certain operations.
In addition to arithmetic operations, ``pd.NA`` also propagates as "missing"
or "unknown" in comparison operations:

.. ipython:: python

    np.nan > 1
    pd.NA > 1

For logical operations, ``pd.NA`` follows the rules of the
`three-valued logic <https://en.wikipedia.org/wiki/Three-valued_logic>`__ (or
*Kleene logic*). For example:

.. ipython:: python

    pd.NA | True

For more, see :ref:`NA section <missing_data.NA>` in the user guide on missing
data.


.. _whatsnew_100.string:

Dedicated string data type
^^^^^^^^^^^^^^^^^^^^^^^^^^

We've added :class:`StringDtype`, an extension type dedicated to string data.
Previously, strings were typically stored in object-dtype NumPy arrays. (:issue:`29975`)

.. warning::

   ``StringDtype`` is currently considered experimental. The implementation
   and parts of the API may change without warning.

The ``'string'`` extension type solves several issues with object-dtype NumPy arrays:

1. You can accidentally store a *mixture* of strings and non-strings in an
   ``object`` dtype array. A ``StringArray`` can only store strings.
2. ``object`` dtype breaks dtype-specific operations like :meth:`DataFrame.select_dtypes`.
   There isn't a clear way to select *just* text while excluding non-text,
   but still object-dtype columns.
3. When reading code, the contents of an ``object`` dtype array is less clear
   than ``string``.


.. ipython:: python

   pd.Series(['abc', None, 'def'], dtype=pd.StringDtype())

You can use the alias ``"string"`` as well.

.. ipython:: python

   s = pd.Series(['abc', None, 'def'], dtype="string")
   s

The usual string accessor methods work. Where appropriate, the return type
of the Series or columns of a DataFrame will also have string dtype.

.. ipython:: python

   s.str.upper()
   s.str.split('b', expand=True).dtypes

String accessor methods returning integers will return a value with :class:`Int64Dtype`

.. ipython:: python

   s.str.count("a")

We recommend explicitly using the ``string`` data type when working with strings.
See :ref:`text.types` for more.

.. _whatsnew_100.boolean:

Boolean data type with missing values support
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

We've added :class:`BooleanDtype` / :class:`~arrays.BooleanArray`, an extension
type dedicated to boolean data that can hold missing values. The default
``bool`` data type based on a bool-dtype NumPy array, the column can only hold
``True`` or ``False``, and not missing values. This new :class:`~arrays.BooleanArray`
can store missing values as well by keeping track of this in a separate mask.
(:issue:`29555`, :issue:`30095`, :issue:`31131`)

.. ipython:: python

   pd.Series([True, False, None], dtype=pd.BooleanDtype())

You can use the alias ``"boolean"`` as well.

.. ipython:: python

   s = pd.Series([True, False, None], dtype="boolean")
   s

.. _whatsnew_100.convert_dtypes:

Method ``convert_dtypes`` to ease use of supported extension dtypes
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

In order to encourage use of the extension dtypes ``StringDtype``,
``BooleanDtype``, ``Int64Dtype``, ``Int32Dtype``, etc., that support ``pd.NA``, the
methods :meth:`DataFrame.convert_dtypes` and :meth:`Series.convert_dtypes`
have been introduced. (:issue:`29752`) (:issue:`30929`)

Example:

.. ipython:: python

   df = pd.DataFrame({'x': ['abc', None, 'def'],
                      'y': [1, 2, np.nan],
                      'z': [True, False, True]})
   df
   df.dtypes

.. ipython:: python

   converted = df.convert_dtypes()
   converted
   converted.dtypes

This is especially useful after reading in data using readers such as :func:`read_csv`
and :func:`read_excel`.
See :ref:`here <missing_data.NA.conversion>` for a description.


.. _whatsnew_100.enhancements.other:

Other enhancements
~~~~~~~~~~~~~~~~~~

- :meth:`DataFrame.to_string` added the ``max_colwidth`` parameter to control when wide columns are truncated (:issue:`9784`)
- Added the ``na_value`` argument to :meth:`Series.to_numpy`, :meth:`Index.to_numpy` and :meth:`DataFrame.to_numpy` to control the value used for missing data (:issue:`30322`)
- :meth:`MultiIndex.from_product` infers level names from inputs if not explicitly provided (:issue:`27292`)
- :meth:`DataFrame.to_latex` now accepts ``caption`` and ``label`` arguments (:issue:`25436`)
- DataFrames with :ref:`nullable integer <integer_na>`, the :ref:`new string dtype <text.types>`
  and period data type can now be converted to ``pyarrow`` (>=0.15.0), which means that it is
  supported in writing to the Parquet file format when using the ``pyarrow`` engine (:issue:`28368`).
  Full roundtrip to parquet (writing and reading back in with :meth:`~DataFrame.to_parquet` / :func:`read_parquet`)
  is supported starting with pyarrow >= 0.16 (:issue:`20612`).
- :func:`to_parquet` now appropriately handles the ``schema`` argument for user defined schemas in the pyarrow engine. (:issue:`30270`)
- :meth:`DataFrame.to_json` now accepts an ``indent`` integer argument to enable pretty printing of JSON output (:issue:`12004`)
- :meth:`read_stata` can read Stata 119 dta files. (:issue:`28250`)
- Implemented :meth:`.Window.var` and :meth:`.Window.std` functions (:issue:`26597`)
- Added ``encoding`` argument to :meth:`DataFrame.to_string` for non-ascii text (:issue:`28766`)
- Added ``encoding`` argument to :func:`DataFrame.to_html` for non-ascii text (:issue:`28663`)
- :meth:`Styler.background_gradient` now accepts ``vmin`` and ``vmax`` arguments (:issue:`12145`)
- :meth:`Styler.format` added the ``na_rep`` parameter to help format the missing values (:issue:`21527`, :issue:`28358`)
- :func:`read_excel` now can read binary Excel (``.xlsb``) files by passing ``engine='pyxlsb'``. For more details and example usage, see the :ref:`Binary Excel files documentation <io.xlsb>`. Closes :issue:`8540`.
- The ``partition_cols`` argument in :meth:`DataFrame.to_parquet` now accepts a string (:issue:`27117`)
- :func:`pandas.read_json` now parses ``NaN``, ``Infinity`` and ``-Infinity`` (:issue:`12213`)
- DataFrame constructor preserve ``ExtensionArray`` dtype with ``ExtensionArray`` (:issue:`11363`)
- :meth:`DataFrame.sort_values` and :meth:`Series.sort_values` have gained ``ignore_index`` keyword to be able to reset index after sorting (:issue:`30114`)
- :meth:`DataFrame.sort_index` and :meth:`Series.sort_index` have gained ``ignore_index`` keyword to reset index (:issue:`30114`)
- :meth:`DataFrame.drop_duplicates` has gained ``ignore_index`` keyword to reset index (:issue:`30114`)
- Added new writer for exporting Stata dta files in versions 118 and 119, ``StataWriterUTF8``.  These files formats support exporting strings containing Unicode characters. Format 119 supports data sets with more than 32,767 variables (:issue:`23573`, :issue:`30959`)
- :meth:`Series.map` now accepts ``collections.abc.Mapping`` subclasses as a mapper (:issue:`29733`)
- Added an experimental :attr:`~DataFrame.attrs` for storing global metadata about a dataset (:issue:`29062`)
- :meth:`Timestamp.fromisocalendar` is now compatible with python 3.8 and above (:issue:`28115`)
- :meth:`DataFrame.to_pickle` and :func:`read_pickle` now accept URL (:issue:`30163`)



.. ---------------------------------------------------------------------------

.. _whatsnew_100.api_breaking:

Backwards incompatible API changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. _whatsnew_100.api_breaking.MultiIndex._names:

Avoid using names from ``MultiIndex.levels``
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

As part of a larger refactor to :class:`MultiIndex` the level names are now
stored separately from the levels (:issue:`27242`). We recommend using
:attr:`MultiIndex.names` to access the names, and :meth:`Index.set_names`
to update the names.

For backwards compatibility, you can still *access* the names via the levels.

.. ipython:: python

   mi = pd.MultiIndex.from_product([[1, 2], ['a', 'b']], names=['x', 'y'])
   mi.levels[0].name

However, it is no longer possible to *update* the names of the ``MultiIndex``
via the level.

.. ipython:: python
   :okexcept:

   mi.levels[0].name = "new name"
   mi.names

To update, use ``MultiIndex.set_names``, which returns a new ``MultiIndex``.

.. ipython:: python

   mi2 = mi.set_names("new name", level=0)
   mi2.names

New repr for :class:`~pandas.arrays.IntervalArray`
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

:class:`pandas.arrays.IntervalArray` adopts a new ``__repr__`` in accordance with other array classes (:issue:`25022`)

*pandas 0.25.x*

.. code-block:: ipython

   In [1]: pd.arrays.IntervalArray.from_tuples([(0, 1), (2, 3)])
   Out[2]:
   IntervalArray([(0, 1], (2, 3]],
                 closed='right',
                 dtype='interval[int64]')

*pandas 1.0.0*

.. ipython:: python

   pd.arrays.IntervalArray.from_tuples([(0, 1), (2, 3)])

``DataFrame.rename`` now only accepts one positional argument
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

:meth:`DataFrame.rename` would previously accept positional arguments that would lead
to ambiguous or undefined behavior. From pandas 1.0, only the very first argument, which
maps labels to their new names along the default axis, is allowed to be passed by position
(:issue:`29136`).

.. ipython:: python
   :suppress:

   df = pd.DataFrame([[1]])

*pandas 0.25.x*

.. code-block:: ipython

   In [1]: df = pd.DataFrame([[1]])
   In [2]: df.rename({0: 1}, {0: 2})
   Out[2]:
   FutureWarning: ...Use named arguments to resolve ambiguity...
      2
   1  1

*pandas 1.0.0*

.. code-block:: ipython

   In [3]: df.rename({0: 1}, {0: 2})
   Traceback (most recent call last):
   ...
   TypeError: rename() takes from 1 to 2 positional arguments but 3 were given

Note that errors will now be raised when conflicting or potentially ambiguous arguments are provided.

*pandas 0.25.x*

.. code-block:: ipython

   In [4]: df.rename({0: 1}, index={0: 2})
   Out[4]:
      0
   1  1

   In [5]: df.rename(mapper={0: 1}, index={0: 2})
   Out[5]:
      0
   2  1

*pandas 1.0.0*

.. code-block:: ipython

   In [6]: df.rename({0: 1}, index={0: 2})
   Traceback (most recent call last):
   ...
   TypeError: Cannot specify both 'mapper' and any of 'index' or 'columns'

   In [7]: df.rename(mapper={0: 1}, index={0: 2})
   Traceback (most recent call last):
   ...
   TypeError: Cannot specify both 'mapper' and any of 'index' or 'columns'

You can still change the axis along which the first positional argument is applied by
supplying the ``axis`` keyword argument.

.. ipython:: python

   df.rename({0: 1})
   df.rename({0: 1}, axis=1)

If you would like to update both the index and column labels, be sure to use the respective
keywords.

.. ipython:: python

   df.rename(index={0: 1}, columns={0: 2})

Extended verbose info output for :class:`~pandas.DataFrame`
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

:meth:`DataFrame.info` now shows line numbers for the columns summary (:issue:`17304`)

*pandas 0.25.x*

.. code-block:: ipython

   In [1]: df = pd.DataFrame({"int_col": [1, 2, 3],
   ...                    "text_col": ["a", "b", "c"],
   ...                    "float_col": [0.0, 0.1, 0.2]})
   In [2]: df.info(verbose=True)
   <class 'pandas.core.frame.DataFrame'>
   RangeIndex: 3 entries, 0 to 2
   Data columns (total 3 columns):
   int_col      3 non-null int64
   text_col     3 non-null object
   float_col    3 non-null float64
   dtypes: float64(1), int64(1), object(1)
   memory usage: 152.0+ bytes

*pandas 1.0.0*

.. ipython:: python

   df = pd.DataFrame({"int_col": [1, 2, 3],
                      "text_col": ["a", "b", "c"],
                      "float_col": [0.0, 0.1, 0.2]})
   df.info(verbose=True)

:meth:`pandas.array` inference changes
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

:meth:`pandas.array` now infers pandas' new extension types in several cases (:issue:`29791`):

1. String data (including missing values) now returns a :class:`arrays.StringArray`.
2. Integer data (including missing values) now returns a :class:`arrays.IntegerArray`.
3. Boolean data (including missing values) now returns the new :class:`arrays.BooleanArray`

*pandas 0.25.x*

.. code-block:: ipython

   In [1]: pd.array(["a", None])
   Out[1]:
   <PandasArray>
   ['a', None]
   Length: 2, dtype: object

   In [2]: pd.array([1, None])
   Out[2]:
   <PandasArray>
   [1, None]
   Length: 2, dtype: object


*pandas 1.0.0*

.. ipython:: python

   pd.array(["a", None])
   pd.array([1, None])

As a reminder, you can specify the ``dtype`` to disable all inference.

:class:`arrays.IntegerArray` now uses :attr:`pandas.NA`
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

:class:`arrays.IntegerArray` now uses :attr:`pandas.NA` rather than
:attr:`numpy.nan` as its missing value marker (:issue:`29964`).

*pandas 0.25.x*

.. code-block:: ipython

   In [1]: a = pd.array([1, 2, None], dtype="Int64")
   In [2]: a
   Out[2]:
   <IntegerArray>
   [1, 2, NaN]
   Length: 3, dtype: Int64

   In [3]: a[2]
   Out[3]:
   nan

*pandas 1.0.0*

.. ipython:: python

   a = pd.array([1, 2, None], dtype="Int64")
   a
   a[2]

This has a few API-breaking consequences.

**Converting to a NumPy ndarray**

When converting to a NumPy array missing values will be ``pd.NA``, which cannot
be converted to a float. So calling ``np.asarray(integer_array, dtype="float")``
will now raise.

*pandas 0.25.x*

.. code-block:: ipython

    In [1]: np.asarray(a, dtype="float")
    Out[1]:
    array([ 1.,  2., nan])

*pandas 1.0.0*

.. ipython:: python
   :okexcept:

   np.asarray(a, dtype="float")

Use :meth:`arrays.IntegerArray.to_numpy` with an explicit ``na_value`` instead.

.. ipython:: python

   a.to_numpy(dtype="float", na_value=np.nan)

**Reductions can return** ``pd.NA``

When performing a reduction such as a sum with ``skipna=False``, the result
will now be ``pd.NA`` instead of ``np.nan`` in presence of missing values
(:issue:`30958`).

*pandas 0.25.x*

.. code-block:: ipython

    In [1]: pd.Series(a).sum(skipna=False)
    Out[1]:
    nan

*pandas 1.0.0*

.. ipython:: python

   pd.Series(a).sum(skipna=False)

**value_counts returns a nullable integer dtype**

:meth:`Series.value_counts` with a nullable integer dtype now returns a nullable
integer dtype for the values.

*pandas 0.25.x*

.. code-block:: ipython

   In [1]: pd.Series([2, 1, 1, None], dtype="Int64").value_counts().dtype
   Out[1]:
   dtype('int64')

*pandas 1.0.0*

.. ipython:: python

   pd.Series([2, 1, 1, None], dtype="Int64").value_counts().dtype

See :ref:`missing_data.NA` for more on the differences between :attr:`pandas.NA`
and :attr:`numpy.nan`.

:class:`arrays.IntegerArray` comparisons return :class:`arrays.BooleanArray`
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Comparison operations on a :class:`arrays.IntegerArray` now returns a
:class:`arrays.BooleanArray` rather than a NumPy array (:issue:`29964`).

*pandas 0.25.x*

.. code-block:: ipython

   In [1]: a = pd.array([1, 2, None], dtype="Int64")
   In [2]: a
   Out[2]:
   <IntegerArray>
   [1, 2, NaN]
   Length: 3, dtype: Int64

   In [3]: a > 1
   Out[3]:
   array([False,  True, False])

*pandas 1.0.0*

.. ipython:: python

   a = pd.array([1, 2, None], dtype="Int64")
   a > 1

Note that missing values now propagate, rather than always comparing unequal
like :attr:`numpy.nan`. See :ref:`missing_data.NA` for more.

By default :meth:`Categorical.min` now returns the minimum instead of np.nan
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

When :class:`Categorical` contains ``np.nan``,
:meth:`Categorical.min` no longer return ``np.nan`` by default (skipna=True) (:issue:`25303`)

*pandas 0.25.x*

.. code-block:: ipython

   In [1]: pd.Categorical([1, 2, np.nan], ordered=True).min()
   Out[1]: nan


*pandas 1.0.0*

.. ipython:: python

   pd.Categorical([1, 2, np.nan], ordered=True).min()


Default dtype of empty :class:`pandas.Series`
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Initialising an empty :class:`pandas.Series` without specifying a dtype will raise a ``DeprecationWarning`` now
(:issue:`17261`). The default dtype will change from ``float64`` to ``object`` in future releases so that it is
consistent with the behaviour of :class:`DataFrame` and :class:`Index`.

*pandas 1.0.0*

.. code-block:: ipython

   In [1]: pd.Series()
   Out[2]:
   DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.
   Series([], dtype: float64)

Result dtype inference changes for resample operations
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The rules for the result dtype in :meth:`DataFrame.resample` aggregations have changed for extension types (:issue:`31359`).
Previously, pandas would attempt to convert the result back to the original dtype, falling back to the usual
inference rules if that was not possible. Now, pandas will only return a result of the original dtype if the
scalar values in the result are instances of the extension dtype's scalar type.

.. ipython:: python

   df = pd.DataFrame({"A": ['a', 'b']}, dtype='category',
                     index=pd.date_range('2000', periods=2))
   df


*pandas 0.25.x*

.. code-block:: ipython

   In [1]> df.resample("2D").agg(lambda x: 'a').A.dtype
   Out[1]:
   CategoricalDtype(categories=['a', 'b'], ordered=False)

*pandas 1.0.0*

.. ipython:: python

   df.resample("2D").agg(lambda x: 'a').A.dtype

This fixes an inconsistency between ``resample`` and ``groupby``.
This also fixes a potential bug, where the **values** of the result might change
depending on how the results are cast back to the original dtype.

*pandas 0.25.x*

.. code-block:: ipython

   In [1] df.resample("2D").agg(lambda x: 'c')
   Out[1]:

        A
   0  NaN

*pandas 1.0.0*

.. ipython:: python

   df.resample("2D").agg(lambda x: 'c')


.. _whatsnew_100.api_breaking.python:

Increased minimum version for Python
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

pandas 1.0.0 supports Python 3.6.1 and higher (:issue:`29212`).

.. _whatsnew_100.api_breaking.deps:

Increased minimum versions for dependencies
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Some minimum supported versions of dependencies were updated (:issue:`29766`, :issue:`29723`).
If installed, we now require:

+-----------------+-----------------+----------+---------+
| Package         | Minimum Version | Required | Changed |
+=================+=================+==========+=========+
| numpy           | 1.13.3          |    X     |         |
+-----------------+-----------------+----------+---------+
| pytz            | 2015.4          |    X     |         |
+-----------------+-----------------+----------+---------+
| python-dateutil | 2.6.1           |    X     |         |
+-----------------+-----------------+----------+---------+
| bottleneck      | 1.2.1           |          |         |
+-----------------+-----------------+----------+---------+
| numexpr         | 2.6.2           |          |         |
+-----------------+-----------------+----------+---------+
| pytest (dev)    | 4.0.2           |          |         |
+-----------------+-----------------+----------+---------+

For `optional libraries <https://pandas.pydata.org/docs/getting_started/install.html>`_ the general recommendation is to use the latest version.
The following table lists the lowest version per library that is currently being tested throughout the development of pandas.
Optional libraries below the lowest tested version may still work, but are not considered supported.

+-----------------+-----------------+---------+
| Package         | Minimum Version | Changed |
+=================+=================+=========+
| beautifulsoup4  | 4.6.0           |         |
+-----------------+-----------------+---------+
| fastparquet     | 0.3.2           |    X    |
+-----------------+-----------------+---------+
| gcsfs           | 0.2.2           |         |
+-----------------+-----------------+---------+
| lxml            | 3.8.0           |         |
+-----------------+-----------------+---------+
| matplotlib      | 2.2.2           |         |
+-----------------+-----------------+---------+
| numba           | 0.46.0          |    X    |
+-----------------+-----------------+---------+
| openpyxl        | 2.5.7           |    X    |
+-----------------+-----------------+---------+
| pyarrow         | 0.13.0          |    X    |
+-----------------+-----------------+---------+
| pymysql         | 0.7.1           |         |
+-----------------+-----------------+---------+
| pytables        | 3.4.2           |         |
+-----------------+-----------------+---------+
| s3fs            | 0.3.0           |    X    |
+-----------------+-----------------+---------+
| scipy           | 0.19.0          |         |
+-----------------+-----------------+---------+
| sqlalchemy      | 1.1.4           |         |
+-----------------+-----------------+---------+
| xarray          | 0.8.2           |         |
+-----------------+-----------------+---------+
| xlrd            | 1.1.0           |         |
+-----------------+-----------------+---------+
| xlsxwriter      | 0.9.8           |         |
+-----------------+-----------------+---------+
| xlwt            | 1.2.0           |         |
+-----------------+-----------------+---------+

See :ref:`install.dependencies` and :ref:`install.optional_dependencies` for more.

Build changes
^^^^^^^^^^^^^

pandas has added a `pyproject.toml <https://www.python.org/dev/peps/pep-0517/>`_ file and will no longer include
cythonized files in the source distribution uploaded to PyPI (:issue:`28341`, :issue:`20775`). If you're installing
a built distribution (wheel) or via conda, this shouldn't have any effect on you. If you're building pandas from
source, you should no longer need to install Cython into your build environment before calling ``pip install pandas``.


.. _whatsnew_100.api.other:

Other API changes
^^^^^^^^^^^^^^^^^

- :meth:`.DataFrameGroupBy.transform` and :meth:`.SeriesGroupBy.transform` now raises on invalid operation names (:issue:`27489`)
- :meth:`pandas.api.types.infer_dtype` will now return "integer-na" for integer and ``np.nan`` mix (:issue:`27283`)
- :meth:`MultiIndex.from_arrays` will no longer infer names from arrays if ``names=None`` is explicitly provided (:issue:`27292`)
- In order to improve tab-completion, pandas does not include most deprecated attributes when introspecting a pandas object using ``dir`` (e.g. ``dir(df)``).
  To see which attributes are excluded, see an object's ``_deprecations`` attribute, for example ``pd.DataFrame._deprecations`` (:issue:`28805`).
- The returned dtype of :func:`unique` now matches the input dtype. (:issue:`27874`)
- Changed the default configuration value for ``options.matplotlib.register_converters`` from ``True`` to ``"auto"`` (:issue:`18720`).
  Now, pandas custom formatters will only be applied to plots created by pandas, through :meth:`~DataFrame.plot`.
  Previously, pandas' formatters would be applied to all plots created *after* a :meth:`~DataFrame.plot`.
  See :ref:`units registration <whatsnew_100.matplotlib_units>` for more.
- :meth:`Series.dropna` has dropped its ``**kwargs`` argument in favor of a single ``how`` parameter.
  Supplying anything else than ``how`` to ``**kwargs`` raised a ``TypeError`` previously (:issue:`29388`)
- When testing pandas, the new minimum required version of pytest is 5.0.1 (:issue:`29664`)
- :meth:`Series.str.__iter__` was deprecated and will be removed in future releases (:issue:`28277`).
- Added ``<NA>`` to the list of default NA values for :meth:`read_csv` (:issue:`30821`)

.. _whatsnew_100.api.documentation:

Documentation improvements
^^^^^^^^^^^^^^^^^^^^^^^^^^

- Added new section on :ref:`scale` (:issue:`28315`).
- Added sub-section on :ref:`io.query_multi` for HDF5 datasets (:issue:`28791`).

.. ---------------------------------------------------------------------------

.. _whatsnew_100.deprecations:

Deprecations
~~~~~~~~~~~~

- :meth:`Series.item` and :meth:`Index.item` have been _undeprecated_ (:issue:`29250`)
- ``Index.set_value`` has been deprecated. For a given index ``idx``, array ``arr``,
  value in ``idx`` of ``idx_val`` and a new value of ``val``, ``idx.set_value(arr, idx_val, val)``
  is equivalent to ``arr[idx.get_loc(idx_val)] = val``, which should be used instead (:issue:`28621`).
- :func:`is_extension_type` is deprecated, :func:`is_extension_array_dtype` should be used instead (:issue:`29457`)
- :func:`eval` keyword argument "truediv" is deprecated and will be removed in a future version (:issue:`29812`)
- :meth:`DateOffset.isAnchored` and :meth:`DatetOffset.onOffset` are deprecated and will be removed in a future version, use :meth:`DateOffset.is_anchored` and :meth:`DateOffset.is_on_offset` instead (:issue:`30340`)
- ``pandas.tseries.frequencies.get_offset`` is deprecated and will be removed in a future version, use ``pandas.tseries.frequencies.to_offset`` instead (:issue:`4205`)
- :meth:`Categorical.take_nd` and :meth:`CategoricalIndex.take_nd` are deprecated, use :meth:`Categorical.take` and :meth:`CategoricalIndex.take` instead (:issue:`27745`)
- The parameter ``numeric_only`` of :meth:`Categorical.min` and :meth:`Categorical.max` is deprecated and replaced with ``skipna`` (:issue:`25303`)
- The parameter ``label`` in :func:`lreshape` has been deprecated and will be removed in a future version (:issue:`29742`)
- ``pandas.core.index`` has been deprecated and will be removed in a future version, the public classes are available in the top-level namespace (:issue:`19711`)
- :func:`pandas.json_normalize` is now exposed in the top-level namespace.
  Usage of ``json_normalize`` as ``pandas.io.json.json_normalize`` is now deprecated and
  it is recommended to use ``json_normalize`` as :func:`pandas.json_normalize` instead (:issue:`27586`).
- The ``numpy`` argument of :meth:`pandas.read_json` is deprecated (:issue:`28512`).
- :meth:`DataFrame.to_stata`, :meth:`DataFrame.to_feather`, and :meth:`DataFrame.to_parquet` argument "fname" is deprecated, use "path" instead (:issue:`23574`)
- The deprecated internal attributes ``_start``, ``_stop`` and ``_step`` of :class:`RangeIndex` now raise a ``FutureWarning`` instead of a ``DeprecationWarning`` (:issue:`26581`)
- The ``pandas.util.testing`` module has been deprecated. Use the public API in ``pandas.testing`` documented at :ref:`api.general.testing` (:issue:`16232`).
- ``pandas.SparseArray`` has been deprecated.  Use ``pandas.arrays.SparseArray`` (:class:`arrays.SparseArray`) instead. (:issue:`30642`)
- The parameter ``is_copy`` of :meth:`Series.take` and :meth:`DataFrame.take` has been deprecated and will be removed in a future version. (:issue:`27357`)
- Support for multi-dimensional indexing (e.g. ``index[:, None]``) on a :class:`Index` is deprecated and will be removed in a future version, convert to a numpy array before indexing instead (:issue:`30588`)
- The ``pandas.np`` submodule is now deprecated. Import numpy directly instead (:issue:`30296`)
- The ``pandas.datetime`` class is now deprecated. Import from ``datetime`` instead (:issue:`30610`)
- :class:`~DataFrame.diff` will raise a ``TypeError`` rather than implicitly losing the dtype of extension types in the future. Convert to the correct dtype before calling ``diff`` instead (:issue:`31025`)

**Selecting Columns from a Grouped DataFrame**

When selecting columns from a :class:`DataFrameGroupBy` object, passing individual keys (or a tuple of keys) inside single brackets is deprecated,
a list of items should be used instead. (:issue:`23566`) For example:

.. code-block:: ipython

    df = pd.DataFrame({
        "A": ["foo", "bar", "foo", "bar", "foo", "bar", "foo", "foo"],
        "B": np.random.randn(8),
        "C": np.random.randn(8),
    })
    g = df.groupby('A')

    # single key, returns SeriesGroupBy
    g['B']

    # tuple of single key, returns SeriesGroupBy
    g[('B',)]

    # tuple of multiple keys, returns DataFrameGroupBy, raises FutureWarning
    g[('B', 'C')]

    # multiple keys passed directly, returns DataFrameGroupBy, raises FutureWarning
    # (implicitly converts the passed strings into a single tuple)
    g['B', 'C']

    # proper way, returns DataFrameGroupBy
    g[['B', 'C']]

.. ---------------------------------------------------------------------------

.. _whatsnew_100.prior_deprecations:

Removal of prior version deprecations/changes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

**Removed SparseSeries and SparseDataFrame**

``SparseSeries``, ``SparseDataFrame`` and the ``DataFrame.to_sparse`` method
have been removed (:issue:`28425`). We recommend using a ``Series`` or
``DataFrame`` with sparse values instead.

.. _whatsnew_100.matplotlib_units:

**Matplotlib unit registration**

Previously, pandas would register converters with matplotlib as a side effect of importing pandas (:issue:`18720`).
This changed the output of plots made via matplotlib plots after pandas was imported, even if you were using
matplotlib directly rather than :meth:`~DataFrame.plot`.

To use pandas formatters with a matplotlib plot, specify

.. code-block:: ipython

   In [1]: import pandas as pd
   In [2]: pd.options.plotting.matplotlib.register_converters = True

Note that plots created by :meth:`DataFrame.plot` and :meth:`Series.plot` *do* register the converters
automatically. The only behavior change is when plotting a date-like object via ``matplotlib.pyplot.plot``
or ``matplotlib.Axes.plot``. See :ref:`plotting.formatters` for more.

**Other removals**

- Removed the previously deprecated keyword "index" from :func:`read_stata`, :class:`StataReader`, and :meth:`StataReader.read`, use "index_col" instead (:issue:`17328`)
- Removed ``StataReader.data`` method, use :meth:`StataReader.read` instead (:issue:`9493`)
- Removed ``pandas.plotting._matplotlib.tsplot``, use :meth:`Series.plot` instead (:issue:`19980`)
- ``pandas.tseries.converter.register`` has been moved to :func:`pandas.plotting.register_matplotlib_converters` (:issue:`18307`)
- :meth:`Series.plot` no longer accepts positional arguments, pass keyword arguments instead (:issue:`30003`)
- :meth:`DataFrame.hist` and :meth:`Series.hist` no longer allows ``figsize="default"``, specify figure size by passinig a tuple instead (:issue:`30003`)
- Floordiv of integer-dtyped array by :class:`Timedelta` now raises ``TypeError`` (:issue:`21036`)
- :class:`TimedeltaIndex` and :class:`DatetimeIndex` no longer accept non-nanosecond dtype strings like "timedelta64" or "datetime64", use "timedelta64[ns]" and "datetime64[ns]" instead (:issue:`24806`)
- Changed the default "skipna" argument in :func:`pandas.api.types.infer_dtype` from ``False`` to ``True`` (:issue:`24050`)
- Removed ``Series.ix`` and ``DataFrame.ix`` (:issue:`26438`)
- Removed ``Index.summary`` (:issue:`18217`)
- Removed the previously deprecated keyword "fastpath" from the :class:`Index` constructor (:issue:`23110`)
- Removed ``Series.get_value``, ``Series.set_value``, ``DataFrame.get_value``, ``DataFrame.set_value`` (:issue:`17739`)
- Removed ``Series.compound`` and ``DataFrame.compound`` (:issue:`26405`)
- Changed the default "inplace" argument in :meth:`DataFrame.set_index` and :meth:`Series.set_axis` from ``None`` to ``False`` (:issue:`27600`)
- Removed ``Series.cat.categorical``, ``Series.cat.index``, ``Series.cat.name`` (:issue:`24751`)
- Removed the previously deprecated keyword "box" from :func:`to_datetime` and :func:`to_timedelta`; in addition these now always returns :class:`DatetimeIndex`, :class:`TimedeltaIndex`, :class:`Index`, :class:`Series`, or :class:`DataFrame` (:issue:`24486`)
- :func:`to_timedelta`, :class:`Timedelta`, and :class:`TimedeltaIndex` no longer allow "M", "y", or "Y" for the "unit" argument (:issue:`23264`)
- Removed the previously deprecated keyword "time_rule" from (non-public) ``offsets.generate_range``, which has been moved to :func:`core.arrays._ranges.generate_range` (:issue:`24157`)
- :meth:`DataFrame.loc` or :meth:`Series.loc` with listlike indexers and missing labels will no longer reindex (:issue:`17295`)
- :meth:`DataFrame.to_excel` and :meth:`Series.to_excel` with non-existent columns will no longer reindex (:issue:`17295`)
- Removed the previously deprecated keyword "join_axes" from :func:`concat`; use ``reindex_like`` on the result instead (:issue:`22318`)
- Removed the previously deprecated keyword "by" from :meth:`DataFrame.sort_index`, use :meth:`DataFrame.sort_values` instead (:issue:`10726`)
- Removed support for nested renaming in :meth:`DataFrame.aggregate`, :meth:`Series.aggregate`, :meth:`core.groupby.DataFrameGroupBy.aggregate`, :meth:`core.groupby.SeriesGroupBy.aggregate`, :meth:`core.window.rolling.Rolling.aggregate` (:issue:`18529`)
- Passing ``datetime64`` data to :class:`TimedeltaIndex` or ``timedelta64`` data to ``DatetimeIndex`` now raises ``TypeError`` (:issue:`23539`, :issue:`23937`)
- Passing ``int64`` values to :class:`DatetimeIndex` and a timezone now interprets the values as nanosecond timestamps in UTC, not wall times in the given timezone (:issue:`24559`)
- A tuple passed to :meth:`DataFrame.groupby` is now exclusively treated as a single key (:issue:`18314`)
- Removed ``Index.contains``, use ``key in index`` instead (:issue:`30103`)
- Addition and subtraction of ``int`` or integer-arrays is no longer allowed in :class:`Timestamp`, :class:`DatetimeIndex`, :class:`TimedeltaIndex`, use ``obj + n * obj.freq`` instead of ``obj + n`` (:issue:`22535`)
- Removed ``Series.ptp`` (:issue:`21614`)
- Removed ``Series.from_array`` (:issue:`18258`)
- Removed ``DataFrame.from_items`` (:issue:`18458`)
- Removed ``DataFrame.as_matrix``, ``Series.as_matrix`` (:issue:`18458`)
- Removed ``Series.asobject`` (:issue:`18477`)
- Removed ``DataFrame.as_blocks``, ``Series.as_blocks``, ``DataFrame.blocks``, ``Series.blocks`` (:issue:`17656`)
- :meth:`pandas.Series.str.cat` now defaults to aligning ``others``, using ``join='left'`` (:issue:`27611`)
- :meth:`pandas.Series.str.cat` does not accept list-likes *within* list-likes anymore (:issue:`27611`)
- :meth:`Series.where` with ``Categorical`` dtype (or :meth:`DataFrame.where` with ``Categorical`` column) no longer allows setting new categories (:issue:`24114`)
- Removed the previously deprecated keywords "start", "end", and "periods" from the :class:`DatetimeIndex`, :class:`TimedeltaIndex`, and :class:`PeriodIndex` constructors; use :func:`date_range`, :func:`timedelta_range`, and :func:`period_range` instead (:issue:`23919`)
- Removed the previously deprecated keyword "verify_integrity" from the :class:`DatetimeIndex` and :class:`TimedeltaIndex` constructors (:issue:`23919`)
- Removed the previously deprecated keyword "fastpath" from ``pandas.core.internals.blocks.make_block`` (:issue:`19265`)
- Removed the previously deprecated keyword "dtype" from :meth:`Block.make_block_same_class` (:issue:`19434`)
- Removed ``ExtensionArray._formatting_values``. Use :attr:`ExtensionArray._formatter` instead. (:issue:`23601`)
- Removed ``MultiIndex.to_hierarchical`` (:issue:`21613`)
- Removed ``MultiIndex.labels``, use :attr:`MultiIndex.codes` instead (:issue:`23752`)
- Removed the previously deprecated keyword "labels" from the :class:`MultiIndex` constructor, use "codes" instead (:issue:`23752`)
- Removed ``MultiIndex.set_labels``, use :meth:`MultiIndex.set_codes` instead (:issue:`23752`)
- Removed the previously deprecated keyword "labels" from :meth:`MultiIndex.set_codes`, :meth:`MultiIndex.copy`, :meth:`MultiIndex.drop`, use "codes" instead (:issue:`23752`)
- Removed support for legacy HDF5 formats (:issue:`29787`)
- Passing a dtype alias (e.g. 'datetime64[ns, UTC]') to :class:`DatetimeTZDtype` is no longer allowed, use :meth:`DatetimeTZDtype.construct_from_string` instead (:issue:`23990`)
- Removed the previously deprecated keyword "skip_footer" from :func:`read_excel`; use "skipfooter" instead (:issue:`18836`)
- :func:`read_excel` no longer allows an integer value for the parameter ``usecols``, instead pass a list of integers from 0 to ``usecols`` inclusive (:issue:`23635`)
- Removed the previously deprecated keyword "convert_datetime64" from :meth:`DataFrame.to_records` (:issue:`18902`)
- Removed ``IntervalIndex.from_intervals`` in favor of the :class:`IntervalIndex` constructor (:issue:`19263`)
- Changed the default "keep_tz" argument in :meth:`DatetimeIndex.to_series` from ``None`` to ``True`` (:issue:`23739`)
- Removed ``api.types.is_period`` and ``api.types.is_datetimetz`` (:issue:`23917`)
- Ability to read pickles containing :class:`Categorical` instances created with pre-0.16 version of pandas has been removed (:issue:`27538`)
- Removed ``pandas.tseries.plotting.tsplot`` (:issue:`18627`)
- Removed the previously deprecated keywords "reduce" and "broadcast" from :meth:`DataFrame.apply` (:issue:`18577`)
- Removed the previously deprecated ``assert_raises_regex`` function in ``pandas._testing`` (:issue:`29174`)
- Removed the previously deprecated ``FrozenNDArray`` class in ``pandas.core.indexes.frozen`` (:issue:`29335`)
- Removed the previously deprecated keyword "nthreads" from :func:`read_feather`, use "use_threads" instead (:issue:`23053`)
- Removed ``Index.is_lexsorted_for_tuple`` (:issue:`29305`)
- Removed support for nested renaming in :meth:`DataFrame.aggregate`, :meth:`Series.aggregate`, :meth:`core.groupby.DataFrameGroupBy.aggregate`, :meth:`core.groupby.SeriesGroupBy.aggregate`, :meth:`core.window.rolling.Rolling.aggregate` (:issue:`29608`)
- Removed ``Series.valid``; use :meth:`Series.dropna` instead (:issue:`18800`)
- Removed ``DataFrame.is_copy``, ``Series.is_copy`` (:issue:`18812`)
- Removed ``DataFrame.get_ftype_counts``, ``Series.get_ftype_counts`` (:issue:`18243`)
- Removed ``DataFrame.ftypes``, ``Series.ftypes``, ``Series.ftype`` (:issue:`26744`)
- Removed ``Index.get_duplicates``, use ``idx[idx.duplicated()].unique()`` instead (:issue:`20239`)
- Removed ``Series.clip_upper``, ``Series.clip_lower``, ``DataFrame.clip_upper``, ``DataFrame.clip_lower`` (:issue:`24203`)
- Removed the ability to alter :attr:`DatetimeIndex.freq`, :attr:`TimedeltaIndex.freq`, or :attr:`PeriodIndex.freq` (:issue:`20772`)
- Removed ``DatetimeIndex.offset`` (:issue:`20730`)
- Removed ``DatetimeIndex.asobject``, ``TimedeltaIndex.asobject``, ``PeriodIndex.asobject``, use ``astype(object)`` instead (:issue:`29801`)
- Removed the previously deprecated keyword "order" from :func:`factorize` (:issue:`19751`)
- Removed the previously deprecated keyword "encoding" from :func:`read_stata` and :meth:`DataFrame.to_stata` (:issue:`21400`)
- Changed the default "sort" argument in :func:`concat` from ``None`` to ``False`` (:issue:`20613`)
- Removed the previously deprecated keyword "raise_conflict" from :meth:`DataFrame.update`, use "errors" instead (:issue:`23585`)
- Removed the previously deprecated keyword "n" from :meth:`DatetimeIndex.shift`, :meth:`TimedeltaIndex.shift`, :meth:`PeriodIndex.shift`, use "periods" instead (:issue:`22458`)
- Removed the previously deprecated keywords "how", "fill_method", and "limit" from :meth:`DataFrame.resample` (:issue:`30139`)
- Passing an integer to :meth:`Series.fillna` or :meth:`DataFrame.fillna` with ``timedelta64[ns]`` dtype now raises ``TypeError`` (:issue:`24694`)
- Passing multiple axes to :meth:`DataFrame.dropna` is no longer supported (:issue:`20995`)
- Removed ``Series.nonzero``, use ``to_numpy().nonzero()`` instead (:issue:`24048`)
- Passing floating dtype ``codes`` to :meth:`Categorical.from_codes` is no longer supported, pass ``codes.astype(np.int64)`` instead (:issue:`21775`)
- Removed the previously deprecated keyword "pat" from :meth:`Series.str.partition` and :meth:`Series.str.rpartition`, use "sep" instead (:issue:`23767`)
- Removed ``Series.put`` (:issue:`27106`)
- Removed ``Series.real``, ``Series.imag`` (:issue:`27106`)
- Removed ``Series.to_dense``, ``DataFrame.to_dense`` (:issue:`26684`)
- Removed ``Index.dtype_str``, use ``str(index.dtype)`` instead (:issue:`27106`)
- :meth:`Categorical.ravel` returns a :class:`Categorical` instead of a ``ndarray`` (:issue:`27199`)
- The 'outer' method on Numpy ufuncs, e.g. ``np.subtract.outer`` operating on :class:`Series` objects is no longer supported, and will raise ``NotImplementedError`` (:issue:`27198`)
- Removed ``Series.get_dtype_counts`` and ``DataFrame.get_dtype_counts`` (:issue:`27145`)
- Changed the default "fill_value" argument in :meth:`Categorical.take` from ``True`` to ``False`` (:issue:`20841`)
- Changed the default value for the ``raw`` argument in :func:`Series.rolling().apply() <.Rolling.apply>`, :func:`DataFrame.rolling().apply() <.Rolling.apply>`, :func:`Series.expanding().apply() <.Expanding.apply>`, and :func:`DataFrame.expanding().apply() <.Expanding.apply>` from ``None`` to ``False`` (:issue:`20584`)
- Removed deprecated behavior of :meth:`Series.argmin` and :meth:`Series.argmax`, use :meth:`Series.idxmin` and :meth:`Series.idxmax` for the old behavior (:issue:`16955`)
- Passing a tz-aware ``datetime.datetime`` or :class:`Timestamp` into the :class:`Timestamp` constructor with the ``tz`` argument now raises a ``ValueError`` (:issue:`23621`)
- Removed ``Series.base``, ``Index.base``, ``Categorical.base``, ``Series.flags``, ``Index.flags``, ``PeriodArray.flags``, ``Series.strides``, ``Index.strides``, ``Series.itemsize``, ``Index.itemsize``, ``Series.data``, ``Index.data`` (:issue:`20721`)
- Changed :meth:`Timedelta.resolution` to match the behavior of the standard library ``datetime.timedelta.resolution``, for the old behavior, use :meth:`Timedelta.resolution_string` (:issue:`26839`)
- Removed ``Timestamp.weekday_name``, ``DatetimeIndex.weekday_name``, and ``Series.dt.weekday_name`` (:issue:`18164`)
- Removed the previously deprecated keyword "errors" in :meth:`Timestamp.tz_localize`, :meth:`DatetimeIndex.tz_localize`, and :meth:`Series.tz_localize` (:issue:`22644`)
- Changed the default "ordered" argument in :class:`CategoricalDtype` from ``None`` to ``False`` (:issue:`26336`)
- :meth:`Series.set_axis` and :meth:`DataFrame.set_axis` now require "labels" as the first argument and "axis" as an optional named parameter (:issue:`30089`)
- Removed ``to_msgpack``, ``read_msgpack``, ``DataFrame.to_msgpack``, ``Series.to_msgpack`` (:issue:`27103`)
- Removed ``Series.compress`` (:issue:`21930`)
- Removed the previously deprecated keyword "fill_value" from :meth:`Categorical.fillna`, use "value" instead (:issue:`19269`)
- Removed the previously deprecated keyword "data" from :func:`andrews_curves`, use "frame" instead (:issue:`6956`)
- Removed the previously deprecated keyword "data" from :func:`parallel_coordinates`, use "frame" instead (:issue:`6956`)
- Removed the previously deprecated keyword "colors" from :func:`parallel_coordinates`, use "color" instead (:issue:`6956`)
- Removed the previously deprecated keywords "verbose" and "private_key" from :func:`read_gbq` (:issue:`30200`)
- Calling ``np.array`` and ``np.asarray`` on tz-aware :class:`Series` and :class:`DatetimeIndex` will now return an object array of tz-aware :class:`Timestamp` (:issue:`24596`)
-

.. ---------------------------------------------------------------------------

.. _whatsnew_100.performance:

Performance improvements
~~~~~~~~~~~~~~~~~~~~~~~~

- Performance improvement in :class:`DataFrame` arithmetic and comparison operations with scalars (:issue:`24990`, :issue:`29853`)
- Performance improvement in indexing with a non-unique :class:`IntervalIndex` (:issue:`27489`)
- Performance improvement in :attr:`MultiIndex.is_monotonic` (:issue:`27495`)
- Performance improvement in :func:`cut` when ``bins`` is an :class:`IntervalIndex` (:issue:`27668`)
- Performance improvement when initializing a :class:`DataFrame` using a ``range`` (:issue:`30171`)
- Performance improvement in :meth:`DataFrame.corr` when ``method`` is ``"spearman"`` (:issue:`28139`)
- Performance improvement in :meth:`DataFrame.replace` when provided a list of values to replace (:issue:`28099`)
- Performance improvement in :meth:`DataFrame.select_dtypes` by using vectorization instead of iterating over a loop (:issue:`28317`)
- Performance improvement in :meth:`Categorical.searchsorted` and  :meth:`CategoricalIndex.searchsorted` (:issue:`28795`)
- Performance improvement when comparing a :class:`Categorical` with a scalar and the scalar is not found in the categories (:issue:`29750`)
- Performance improvement when checking if values in a :class:`Categorical` are equal, equal or larger or larger than a given scalar.
  The improvement is not present if checking if the :class:`Categorical` is less than or less than or equal than the scalar (:issue:`29820`)
- Performance improvement in :meth:`Index.equals` and  :meth:`MultiIndex.equals` (:issue:`29134`)
- Performance improvement in :func:`~pandas.api.types.infer_dtype` when ``skipna`` is ``True`` (:issue:`28814`)

.. ---------------------------------------------------------------------------

.. _whatsnew_100.bug_fixes:

Bug fixes
~~~~~~~~~


Categorical
^^^^^^^^^^^

- Added test to assert the :func:`fillna` raises the correct ``ValueError`` message when the value isn't a value from categories (:issue:`13628`)
- Bug in :meth:`Categorical.astype` where ``NaN`` values were handled incorrectly when casting to int (:issue:`28406`)
- :meth:`DataFrame.reindex` with a :class:`CategoricalIndex` would fail when the targets contained duplicates, and wouldn't fail if the source contained duplicates (:issue:`28107`)
- Bug in :meth:`Categorical.astype` not allowing for casting to extension dtypes (:issue:`28668`)
- Bug where :func:`merge` was unable to join on categorical and extension dtype columns (:issue:`28668`)
- :meth:`Categorical.searchsorted` and :meth:`CategoricalIndex.searchsorted` now work on unordered categoricals also (:issue:`21667`)
- Added test to assert roundtripping to parquet with :func:`DataFrame.to_parquet` or :func:`read_parquet` will preserve Categorical dtypes for string types (:issue:`27955`)
- Changed the error message in :meth:`Categorical.remove_categories` to always show the invalid removals as a set (:issue:`28669`)
- Using date accessors on a categorical dtyped :class:`Series` of datetimes was not returning an object of the
  same type as if one used the :meth:`.str.` / :meth:`.dt.` on a :class:`Series` of that type. E.g. when accessing :meth:`Series.dt.tz_localize` on a
  :class:`Categorical` with duplicate entries, the accessor was skipping duplicates (:issue:`27952`)
- Bug in :meth:`DataFrame.replace` and :meth:`Series.replace` that would give incorrect results on categorical data (:issue:`26988`)
- Bug where calling :meth:`Categorical.min` or :meth:`Categorical.max` on an empty Categorical would raise a numpy exception (:issue:`30227`)
- The following methods now also correctly output values for unobserved categories when called through ``groupby(..., observed=False)`` (:issue:`17605`)
  * :meth:`core.groupby.SeriesGroupBy.count`
  * :meth:`core.groupby.SeriesGroupBy.size`
  * :meth:`core.groupby.SeriesGroupBy.nunique`
  * :meth:`core.groupby.SeriesGroupBy.nth`


Datetimelike
^^^^^^^^^^^^
- Bug in :meth:`Series.__setitem__` incorrectly casting ``np.timedelta64("NaT")`` to ``np.datetime64("NaT")`` when inserting into a :class:`Series` with datetime64 dtype (:issue:`27311`)
- Bug in :meth:`Series.dt` property lookups when the underlying data is read-only (:issue:`27529`)
- Bug in ``HDFStore.__getitem__`` incorrectly reading tz attribute created in Python 2 (:issue:`26443`)
- Bug in :func:`to_datetime` where passing arrays of malformed ``str`` with errors="coerce" could incorrectly lead to raising ``ValueError`` (:issue:`28299`)
- Bug in :meth:`core.groupby.SeriesGroupBy.nunique` where ``NaT`` values were interfering with the count of unique values (:issue:`27951`)
- Bug in :class:`Timestamp` subtraction when subtracting a :class:`Timestamp` from a ``np.datetime64`` object incorrectly raising ``TypeError`` (:issue:`28286`)
- Addition and subtraction of integer or integer-dtype arrays with :class:`Timestamp` will now raise ``NullFrequencyError`` instead of ``ValueError`` (:issue:`28268`)
- Bug in :class:`Series` and :class:`DataFrame` with integer dtype failing to raise ``TypeError`` when adding or subtracting a ``np.datetime64`` object (:issue:`28080`)
- Bug in :meth:`Series.astype`, :meth:`Index.astype`, and :meth:`DataFrame.astype` failing to handle ``NaT`` when casting to an integer dtype (:issue:`28492`)
- Bug in :class:`Week` with ``weekday`` incorrectly raising ``AttributeError`` instead of ``TypeError`` when adding or subtracting an invalid type (:issue:`28530`)
- Bug in :class:`DataFrame` arithmetic operations when operating with a :class:`Series` with dtype ``'timedelta64[ns]'`` (:issue:`28049`)
- Bug in :func:`core.groupby.generic.SeriesGroupBy.apply` raising ``ValueError`` when a column in the original DataFrame is a datetime and the column labels are not standard integers (:issue:`28247`)
- Bug in :func:`pandas._config.localization.get_locales` where the ``locales -a`` encodes the locales list as windows-1252 (:issue:`23638`, :issue:`24760`, :issue:`27368`)
- Bug in :meth:`Series.var` failing to raise ``TypeError`` when called with ``timedelta64[ns]`` dtype (:issue:`28289`)
- Bug in :meth:`DatetimeIndex.strftime` and :meth:`Series.dt.strftime` where ``NaT`` was converted to the string ``'NaT'`` instead of ``np.nan`` (:issue:`29578`)
- Bug in masking datetime-like arrays with a boolean mask of an incorrect length not raising an ``IndexError`` (:issue:`30308`)
- Bug in :attr:`Timestamp.resolution` being a property instead of a class attribute (:issue:`29910`)
- Bug in :func:`pandas.to_datetime` when called with ``None`` raising ``TypeError`` instead of returning ``NaT`` (:issue:`30011`)
- Bug in :func:`pandas.to_datetime` failing for ``deque`` objects when using ``cache=True`` (the default) (:issue:`29403`)
- Bug in :meth:`Series.item` with ``datetime64`` or ``timedelta64`` dtype, :meth:`DatetimeIndex.item`, and :meth:`TimedeltaIndex.item` returning an integer instead of a :class:`Timestamp` or :class:`Timedelta` (:issue:`30175`)
- Bug in :class:`DatetimeIndex` addition when adding a non-optimized :class:`DateOffset` incorrectly dropping timezone information (:issue:`30336`)
- Bug in :meth:`DataFrame.drop` where attempting to drop non-existent values from a DatetimeIndex would yield a confusing error message (:issue:`30399`)
- Bug in :meth:`DataFrame.append` would remove the timezone-awareness of new data (:issue:`30238`)
- Bug in :meth:`Series.cummin` and :meth:`Series.cummax` with timezone-aware dtype incorrectly dropping its timezone (:issue:`15553`)
- Bug in :class:`DatetimeArray`, :class:`TimedeltaArray`, and :class:`PeriodArray` where inplace addition and subtraction did not actually operate inplace (:issue:`24115`)
- Bug in :func:`pandas.to_datetime` when called with ``Series`` storing ``IntegerArray`` raising ``TypeError`` instead of returning ``Series`` (:issue:`30050`)
- Bug in :func:`date_range` with custom business hours as ``freq`` and given number of ``periods`` (:issue:`30593`)
- Bug in :class:`PeriodIndex` comparisons with incorrectly casting integers to :class:`Period` objects, inconsistent with the :class:`Period` comparison behavior (:issue:`30722`)
- Bug in :meth:`DatetimeIndex.insert` raising a ``ValueError`` instead of a ``TypeError`` when trying to insert a timezone-aware :class:`Timestamp` into a timezone-naive :class:`DatetimeIndex`, or vice-versa (:issue:`30806`)

Timedelta
^^^^^^^^^
- Bug in subtracting a :class:`TimedeltaIndex` or :class:`TimedeltaArray` from a ``np.datetime64`` object (:issue:`29558`)
-

Timezones
^^^^^^^^^

-


Numeric
^^^^^^^
- Bug in :meth:`DataFrame.quantile` with zero-column :class:`DataFrame` incorrectly raising (:issue:`23925`)
- :class:`DataFrame` flex inequality comparisons methods (:meth:`DataFrame.lt`, :meth:`DataFrame.le`, :meth:`DataFrame.gt`, :meth:`DataFrame.ge`) with object-dtype and ``complex`` entries failing to raise ``TypeError`` like their :class:`Series` counterparts (:issue:`28079`)
- Bug in :class:`DataFrame` logical operations (``&``, ``|``, ``^``) not matching :class:`Series` behavior by filling NA values (:issue:`28741`)
- Bug in :meth:`DataFrame.interpolate` where specifying axis by name references variable before it is assigned (:issue:`29142`)
- Bug in :meth:`Series.var` not computing the right value with a nullable integer dtype series not passing through ddof argument (:issue:`29128`)
- Improved error message when using ``frac`` > 1 and ``replace`` = False (:issue:`27451`)
- Bug in numeric indexes resulted in it being possible to instantiate an :class:`Int64Index`, :class:`UInt64Index`, or :class:`Float64Index` with an invalid dtype (e.g. datetime-like) (:issue:`29539`)
- Bug in :class:`UInt64Index` precision loss while constructing from a list with values in the ``np.uint64`` range (:issue:`29526`)
- Bug in :class:`NumericIndex` construction that caused indexing to fail when integers in the ``np.uint64`` range were used (:issue:`28023`)
- Bug in :class:`NumericIndex` construction that caused :class:`UInt64Index` to be casted to :class:`Float64Index` when integers in the ``np.uint64`` range were used to index a :class:`DataFrame` (:issue:`28279`)
- Bug in :meth:`Series.interpolate` when using method=`index` with an unsorted index, would previously return incorrect results. (:issue:`21037`)
- Bug in :meth:`DataFrame.round` where a :class:`DataFrame` with a :class:`CategoricalIndex` of :class:`IntervalIndex` columns would incorrectly raise a ``TypeError`` (:issue:`30063`)
- Bug in :meth:`Series.pct_change` and :meth:`DataFrame.pct_change` when there are duplicated indices (:issue:`30463`)
- Bug in :class:`DataFrame` cumulative operations (e.g. cumsum, cummax) incorrect casting to object-dtype (:issue:`19296`)
- Bug in :class:`~DataFrame.diff` losing the dtype for extension types (:issue:`30889`)
- Bug in :class:`DataFrame.diff` raising an ``IndexError`` when one of the columns was a nullable integer dtype (:issue:`30967`)

Conversion
^^^^^^^^^^

-

Strings
^^^^^^^

- Calling :meth:`Series.str.isalnum` (and other "ismethods") on an empty ``Series`` would return an ``object`` dtype instead of ``bool`` (:issue:`29624`)
-


Interval
^^^^^^^^

- Bug in :meth:`IntervalIndex.get_indexer` where a :class:`Categorical` or :class:`CategoricalIndex` ``target`` would incorrectly raise a ``TypeError`` (:issue:`30063`)
- Bug in ``pandas.core.dtypes.cast.infer_dtype_from_scalar`` where passing ``pandas_dtype=True`` did not infer :class:`IntervalDtype` (:issue:`30337`)
- Bug in :class:`Series` constructor where constructing a ``Series`` from a ``list`` of :class:`Interval` objects resulted in ``object`` dtype instead of :class:`IntervalDtype` (:issue:`23563`)
- Bug in :class:`IntervalDtype` where the ``kind`` attribute was incorrectly set as ``None`` instead of ``"O"`` (:issue:`30568`)
- Bug in :class:`IntervalIndex`, :class:`~arrays.IntervalArray`, and :class:`Series` with interval data where equality comparisons were incorrect (:issue:`24112`)

Indexing
^^^^^^^^

- Bug in assignment using a reverse slicer (:issue:`26939`)
- Bug in :meth:`DataFrame.explode` would duplicate frame in the presence of duplicates in the index (:issue:`28010`)
- Bug in reindexing a :meth:`PeriodIndex` with another type of index that contained a ``Period`` (:issue:`28323`) (:issue:`28337`)
- Fix assignment of column via ``.loc`` with numpy non-ns datetime type (:issue:`27395`)
- Bug in :meth:`Float64Index.astype` where ``np.inf`` was not handled properly when casting to an integer dtype (:issue:`28475`)
- :meth:`Index.union` could fail when the left contained duplicates (:issue:`28257`)
- Bug when indexing with ``.loc`` where the index was a :class:`CategoricalIndex` with non-string categories didn't work (:issue:`17569`, :issue:`30225`)
- :meth:`Index.get_indexer_non_unique` could fail with ``TypeError`` in some cases, such as when searching for ints in a string index (:issue:`28257`)
- Bug in :meth:`Float64Index.get_loc` incorrectly raising ``TypeError`` instead of ``KeyError`` (:issue:`29189`)
- Bug in :meth:`DataFrame.loc` with incorrect dtype when setting Categorical value in 1-row DataFrame (:issue:`25495`)
- :meth:`MultiIndex.get_loc` can't find missing values when input includes missing values (:issue:`19132`)
- Bug in :meth:`Series.__setitem__` incorrectly assigning values with boolean indexer when the length of new data matches the number of ``True`` values and new data is not a ``Series`` or an ``np.array`` (:issue:`30567`)
- Bug in indexing with a :class:`PeriodIndex` incorrectly accepting integers representing years, use e.g. ``ser.loc["2007"]`` instead of ``ser.loc[2007]`` (:issue:`30763`)

Missing
^^^^^^^

-

MultiIndex
^^^^^^^^^^

- Constructor for :class:`MultiIndex` verifies that the given ``sortorder`` is compatible with the actual ``lexsort_depth``  if ``verify_integrity`` parameter is ``True`` (the default) (:issue:`28735`)
- Series and MultiIndex ``.drop`` with ``MultiIndex`` raise exception if labels not in given in level (:issue:`8594`)
-

IO
^^

- :meth:`read_csv` now accepts binary mode file buffers when using the Python csv engine (:issue:`23779`)
- Bug in :meth:`DataFrame.to_json` where using a Tuple as a column or index value and using ``orient="columns"`` or ``orient="index"`` would produce invalid JSON (:issue:`20500`)
- Improve infinity parsing. :meth:`read_csv` now interprets ``Infinity``, ``+Infinity``, ``-Infinity`` as floating point values (:issue:`10065`)
- Bug in :meth:`DataFrame.to_csv` where values were truncated when the length of ``na_rep`` was shorter than the text input data. (:issue:`25099`)
- Bug in :func:`DataFrame.to_string` where values were truncated using display options instead of outputting the full content (:issue:`9784`)
- Bug in :meth:`DataFrame.to_json` where a datetime column label would not be written out in ISO format with ``orient="table"`` (:issue:`28130`)
- Bug in :func:`DataFrame.to_parquet` where writing to GCS would fail with ``engine='fastparquet'`` if the file did not already exist (:issue:`28326`)
- Bug in :func:`read_hdf` closing stores that it didn't open when Exceptions are raised (:issue:`28699`)
- Bug in :meth:`DataFrame.read_json` where using ``orient="index"`` would not maintain the order (:issue:`28557`)
- Bug in :meth:`DataFrame.to_html` where the length of the ``formatters`` argument was not verified (:issue:`28469`)
- Bug in :meth:`DataFrame.read_excel` with ``engine='ods'`` when ``sheet_name`` argument references a non-existent sheet (:issue:`27676`)
- Bug in :meth:`pandas.io.formats.style.Styler` formatting for floating values not displaying decimals correctly (:issue:`13257`)
- Bug in :meth:`DataFrame.to_html` when using ``formatters=<list>`` and ``max_cols`` together. (:issue:`25955`)
- Bug in :meth:`Styler.background_gradient` not able to work with dtype ``Int64`` (:issue:`28869`)
- Bug in :meth:`DataFrame.to_clipboard` which did not work reliably in ipython (:issue:`22707`)
- Bug in :func:`read_json` where default encoding was not set to ``utf-8`` (:issue:`29565`)
- Bug in :class:`PythonParser` where str and bytes were being mixed when dealing with the decimal field (:issue:`29650`)
- :meth:`read_gbq` now accepts ``progress_bar_type`` to display progress bar while the data downloads. (:issue:`29857`)
- Bug in :func:`pandas.io.json.json_normalize` where a missing value in the location specified by ``record_path`` would raise a ``TypeError`` (:issue:`30148`)
- :func:`read_excel` now accepts binary data (:issue:`15914`)
- Bug in :meth:`read_csv` in which encoding handling was limited to just the string ``utf-16`` for the C engine (:issue:`24130`)

Plotting
^^^^^^^^

- Bug in :meth:`Series.plot` not able to plot boolean values (:issue:`23719`)
- Bug in :meth:`DataFrame.plot` not able to plot when no rows (:issue:`27758`)
- Bug in :meth:`DataFrame.plot` producing incorrect legend markers when plotting multiple series on the same axis (:issue:`18222`)
- Bug in :meth:`DataFrame.plot` when ``kind='box'`` and data contains datetime or timedelta data. These types are now automatically dropped (:issue:`22799`)
- Bug in :meth:`DataFrame.plot.line` and :meth:`DataFrame.plot.area` produce wrong xlim in x-axis (:issue:`27686`, :issue:`25160`, :issue:`24784`)
- Bug where :meth:`DataFrame.boxplot` would not accept a ``color`` parameter like :meth:`DataFrame.plot.box` (:issue:`26214`)
- Bug in the ``xticks`` argument being ignored for :meth:`DataFrame.plot.bar` (:issue:`14119`)
- :func:`set_option` now validates that the plot backend provided to ``'plotting.backend'`` implements the backend when the option is set, rather than when a plot is created (:issue:`28163`)
- :meth:`DataFrame.plot` now allow a ``backend`` keyword argument to allow changing between backends in one session (:issue:`28619`).
- Bug in color validation incorrectly raising for non-color styles (:issue:`29122`).
- Allow :meth:`DataFrame.plot.scatter` to plot ``objects`` and ``datetime`` type data (:issue:`18755`, :issue:`30391`)
- Bug in :meth:`DataFrame.hist`, ``xrot=0`` does not work with ``by`` and subplots (:issue:`30288`).

GroupBy/resample/rolling
^^^^^^^^^^^^^^^^^^^^^^^^

- Bug in :meth:`core.groupby.DataFrameGroupBy.apply` only showing output from a single group when function returns an :class:`Index` (:issue:`28652`)
- Bug in :meth:`DataFrame.groupby` with multiple groups where an ``IndexError`` would be raised if any group contained all NA values (:issue:`20519`)
- Bug in :meth:`.Resampler.size` and :meth:`.Resampler.count` returning wrong dtype when used with an empty :class:`Series` or :class:`DataFrame` (:issue:`28427`)
- Bug in :meth:`DataFrame.rolling` not allowing for rolling over datetimes when ``axis=1`` (:issue:`28192`)
- Bug in :meth:`DataFrame.rolling` not allowing rolling over multi-index levels (:issue:`15584`).
- Bug in :meth:`DataFrame.rolling` not allowing rolling on monotonic decreasing time indexes (:issue:`19248`).
- Bug in :meth:`DataFrame.groupby` not offering selection by column name when ``axis=1`` (:issue:`27614`)
- Bug in :meth:`core.groupby.DataFrameGroupby.agg` not able to use lambda function with named aggregation (:issue:`27519`)
- Bug in :meth:`DataFrame.groupby` losing column name information when grouping by a categorical column (:issue:`28787`)
- Remove error raised due to duplicated input functions in named aggregation in :meth:`DataFrame.groupby` and :meth:`Series.groupby`. Previously error will be raised if the same function is applied on the same column and now it is allowed if new assigned names are different. (:issue:`28426`)
- :meth:`core.groupby.SeriesGroupBy.value_counts` will be able to handle the case even when the :class:`Grouper` makes empty groups (:issue:`28479`)
- Bug in :meth:`core.window.rolling.Rolling.quantile` ignoring ``interpolation`` keyword argument when used within a groupby (:issue:`28779`)
- Bug in :meth:`DataFrame.groupby` where ``any``, ``all``, ``nunique`` and transform functions would incorrectly handle duplicate column labels (:issue:`21668`)
- Bug in :meth:`core.groupby.DataFrameGroupBy.agg` with timezone-aware datetime64 column incorrectly casting results to the original dtype (:issue:`29641`)
- Bug in :meth:`DataFrame.groupby` when using axis=1 and having a single level columns index (:issue:`30208`)
- Bug in :meth:`DataFrame.groupby` when using nunique on axis=1 (:issue:`30253`)
- Bug in :meth:`.DataFrameGroupBy.quantile` and :meth:`.SeriesGroupBy.quantile` with multiple list-like q value and integer column names (:issue:`30289`)
- Bug in :meth:`.DataFrameGroupBy.pct_change` and :meth:`.SeriesGroupBy.pct_change` causes ``TypeError`` when ``fill_method`` is ``None`` (:issue:`30463`)
- Bug in :meth:`Rolling.count` and :meth:`Expanding.count` argument where ``min_periods`` was ignored (:issue:`26996`)

Reshaping
^^^^^^^^^

- Bug in :meth:`DataFrame.apply` that caused incorrect output with empty :class:`DataFrame` (:issue:`28202`, :issue:`21959`)
- Bug in :meth:`DataFrame.stack` not handling non-unique indexes correctly when creating MultiIndex (:issue:`28301`)
- Bug in :meth:`pivot_table` not returning correct type ``float`` when ``margins=True`` and ``aggfunc='mean'`` (:issue:`24893`)
- Bug :func:`merge_asof` could not use :class:`datetime.timedelta` for ``tolerance`` kwarg (:issue:`28098`)
- Bug in :func:`merge`, did not append suffixes correctly with MultiIndex (:issue:`28518`)
- :func:`qcut` and :func:`cut` now handle boolean input (:issue:`20303`)
- Fix to ensure all int dtypes can be used in :func:`merge_asof` when using a tolerance value. Previously every non-int64 type would raise an erroneous ``MergeError`` (:issue:`28870`).
- Better error message in :func:`get_dummies` when ``columns`` isn't a list-like value (:issue:`28383`)
- Bug in :meth:`Index.join` that caused infinite recursion error for mismatched ``MultiIndex`` name orders. (:issue:`25760`, :issue:`28956`)
- Bug :meth:`Series.pct_change` where supplying an anchored frequency would throw a ``ValueError`` (:issue:`28664`)
- Bug where :meth:`DataFrame.equals` returned True incorrectly in some cases when two DataFrames had the same columns in different orders (:issue:`28839`)
- Bug in :meth:`DataFrame.replace` that caused non-numeric replacer's dtype not respected (:issue:`26632`)
- Bug in :func:`melt` where supplying mixed strings and numeric values for ``id_vars`` or ``value_vars`` would incorrectly raise a ``ValueError`` (:issue:`29718`)
- Dtypes are now preserved when transposing a ``DataFrame`` where each column is the same extension dtype (:issue:`30091`)
- Bug in :func:`merge_asof` merging on a tz-aware ``left_index`` and ``right_on`` a tz-aware column (:issue:`29864`)
- Improved error message and docstring in :func:`cut` and :func:`qcut` when ``labels=True`` (:issue:`13318`)
- Bug in missing ``fill_na`` parameter to :meth:`DataFrame.unstack` with list of levels (:issue:`30740`)

Sparse
^^^^^^
- Bug in :class:`SparseDataFrame` arithmetic operations incorrectly casting inputs to float (:issue:`28107`)
- Bug in ``DataFrame.sparse`` returning a ``Series`` when there was a column named ``sparse`` rather than the accessor (:issue:`30758`)
- Fixed :meth:`operator.xor` with a boolean-dtype ``SparseArray``. Now returns a sparse result, rather than object dtype (:issue:`31025`)

ExtensionArray
^^^^^^^^^^^^^^

- Bug in :class:`arrays.PandasArray` when setting a scalar string (:issue:`28118`, :issue:`28150`).
- Bug where nullable integers could not be compared to strings (:issue:`28930`)
- Bug where :class:`DataFrame` constructor raised ``ValueError`` with list-like data and ``dtype`` specified (:issue:`30280`)

Other
^^^^^
- Trying to set the ``display.precision``, ``display.max_rows`` or ``display.max_columns`` using :meth:`set_option` to anything but a ``None`` or a positive int will raise a ``ValueError`` (:issue:`23348`)
- Using :meth:`DataFrame.replace` with overlapping keys in a nested dictionary will no longer raise, now matching the behavior of a flat dictionary (:issue:`27660`)
- :meth:`DataFrame.to_csv` and :meth:`Series.to_csv` now support dicts as ``compression`` argument with key ``'method'`` being the compression method and others as additional compression options when the compression method is ``'zip'``. (:issue:`26023`)
- Bug in :meth:`Series.diff` where a boolean series would incorrectly raise a ``TypeError`` (:issue:`17294`)
- :meth:`Series.append` will no longer raise a ``TypeError`` when passed a tuple of ``Series`` (:issue:`28410`)
- Fix corrupted error message when calling ``pandas.libs._json.encode()`` on a 0d array (:issue:`18878`)
- Backtick quoting in :meth:`DataFrame.query` and :meth:`DataFrame.eval` can now also be used to use invalid identifiers like names that start with a digit, are python keywords, or are using single character operators. (:issue:`27017`)
- Bug in ``pd.core.util.hashing.hash_pandas_object`` where arrays containing tuples were incorrectly treated as non-hashable (:issue:`28969`)
- Bug in :meth:`DataFrame.append` that raised ``IndexError`` when appending with empty list (:issue:`28769`)
- Fix :class:`AbstractHolidayCalendar` to return correct results for
  years after 2030 (now goes up to 2200) (:issue:`27790`)
- Fixed :class:`~arrays.IntegerArray` returning ``inf`` rather than ``NaN`` for operations dividing by ``0`` (:issue:`27398`)
- Fixed ``pow`` operations for :class:`~arrays.IntegerArray` when the other value is ``0`` or ``1`` (:issue:`29997`)
- Bug in :meth:`Series.count` raises if use_inf_as_na is enabled (:issue:`29478`)
- Bug in :class:`Index` where a non-hashable name could be set without raising ``TypeError`` (:issue:`29069`)
- Bug in :class:`DataFrame` constructor when passing a 2D ``ndarray`` and an extension dtype (:issue:`12513`)
- Bug in :meth:`DataFrame.to_csv` when supplied a series with a ``dtype="string"`` and a ``na_rep``, the ``na_rep`` was being truncated to 2 characters. (:issue:`29975`)
- Bug where :meth:`DataFrame.itertuples` would incorrectly determine whether or not namedtuples could be used for dataframes of 255 columns (:issue:`28282`)
- Handle nested NumPy ``object`` arrays in :func:`testing.assert_series_equal` for ExtensionArray implementations (:issue:`30841`)
- Bug in :class:`Index` constructor incorrectly allowing 2-dimensional input arrays (:issue:`13601`, :issue:`27125`)

.. ---------------------------------------------------------------------------

.. _whatsnew_100.contributors:

Contributors
~~~~~~~~~~~~

.. contributors:: v0.25.3..v1.0.0