1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
|
.. _options:
.. currentmodule:: pandas
.. ipython:: python
:suppress:
import pandas as pd
import numpy as np
np.random.seed(123456)
********************
Options and Settings
********************
Overview
--------
pandas has an options system that lets you customize some aspects of its behaviour,
display-related options being those the user is most likely to adjust.
Options have a full "dotted-style", case-insensitive name (e.g. ``display.max_rows``).
You can get/set options directly as attributes of the top-level ``options`` attribute:
.. ipython:: python
import pandas as pd
pd.options.display.max_rows
pd.options.display.max_rows = 999
pd.options.display.max_rows
The API is composed of 5 relevant functions, available directly from the ``pandas``
namespace:
- :func:`~pandas.get_option` / :func:`~pandas.set_option` - get/set the value of a single option.
- :func:`~pandas.reset_option` - reset one or more options to their default value.
- :func:`~pandas.describe_option` - print the descriptions of one or more options.
- :func:`~pandas.option_context` - execute a codeblock with a set of options
that revert to prior settings after execution.
**Note:** Developers can check out `pandas/core/config.py <https://github.com/pandas-dev/pandas/blob/master/pandas/core/config.py>`_ for more information.
All of the functions above accept a regexp pattern (``re.search`` style) as an argument,
and so passing in a substring will work - as long as it is unambiguous:
.. ipython:: python
pd.get_option("display.max_rows")
pd.set_option("display.max_rows",101)
pd.get_option("display.max_rows")
pd.set_option("max_r",102)
pd.get_option("display.max_rows")
The following will **not work** because it matches multiple option names, e.g.
``display.max_colwidth``, ``display.max_rows``, ``display.max_columns``:
.. ipython:: python
:okexcept:
try:
pd.get_option("column")
except KeyError as e:
print(e)
**Note:** Using this form of shorthand may cause your code to break if new options with similar names are added in future versions.
You can get a list of available options and their descriptions with ``describe_option``. When called
with no argument ``describe_option`` will print out the descriptions for all available options.
.. ipython:: python
:suppress:
:okwarning:
pd.reset_option("all")
Getting and Setting Options
---------------------------
As described above, :func:`~pandas.get_option` and :func:`~pandas.set_option`
are available from the pandas namespace. To change an option, call
``set_option('option regex', new_value)``.
.. ipython:: python
pd.get_option('mode.sim_interactive')
pd.set_option('mode.sim_interactive', True)
pd.get_option('mode.sim_interactive')
**Note:** The option 'mode.sim_interactive' is mostly used for debugging purposes.
All options also have a default value, and you can use ``reset_option`` to do just that:
.. ipython:: python
:suppress:
pd.reset_option("display.max_rows")
.. ipython:: python
pd.get_option("display.max_rows")
pd.set_option("display.max_rows",999)
pd.get_option("display.max_rows")
pd.reset_option("display.max_rows")
pd.get_option("display.max_rows")
It's also possible to reset multiple options at once (using a regex):
.. ipython:: python
:okwarning:
pd.reset_option("^display")
``option_context`` context manager has been exposed through
the top-level API, allowing you to execute code with given option values. Option values
are restored automatically when you exit the `with` block:
.. ipython:: python
with pd.option_context("display.max_rows",10,"display.max_columns", 5):
print(pd.get_option("display.max_rows"))
print(pd.get_option("display.max_columns"))
print(pd.get_option("display.max_rows"))
print(pd.get_option("display.max_columns"))
Setting Startup Options in python/ipython Environment
-----------------------------------------------------
Using startup scripts for the python/ipython environment to import pandas and set options makes working with pandas more efficient. To do this, create a .py or .ipy script in the startup directory of the desired profile. An example where the startup folder is in a default ipython profile can be found at:
.. code-block:: none
$IPYTHONDIR/profile_default/startup
More information can be found in the `ipython documentation
<http://ipython.org/ipython-doc/stable/interactive/tutorial.html#startup-files>`__. An example startup script for pandas is displayed below:
.. code-block:: python
import pandas as pd
pd.set_option('display.max_rows', 999)
pd.set_option('precision', 5)
.. _options.frequently_used:
Frequently Used Options
-----------------------
The following is a walkthrough of the more frequently used display options.
``display.max_rows`` and ``display.max_columns`` sets the maximum number
of rows and columns displayed when a frame is pretty-printed. Truncated
lines are replaced by an ellipsis.
.. ipython:: python
df = pd.DataFrame(np.random.randn(7,2))
pd.set_option('max_rows', 7)
df
pd.set_option('max_rows', 5)
df
pd.reset_option('max_rows')
``display.expand_frame_repr`` allows for the representation of
dataframes to stretch across pages, wrapped over the full column vs row-wise.
.. ipython:: python
df = pd.DataFrame(np.random.randn(5,10))
pd.set_option('expand_frame_repr', True)
df
pd.set_option('expand_frame_repr', False)
df
pd.reset_option('expand_frame_repr')
``display.large_repr`` lets you select whether to display dataframes that exceed
``max_columns`` or ``max_rows`` as a truncated frame, or as a summary.
.. ipython:: python
df = pd.DataFrame(np.random.randn(10,10))
pd.set_option('max_rows', 5)
pd.set_option('large_repr', 'truncate')
df
pd.set_option('large_repr', 'info')
df
pd.reset_option('large_repr')
pd.reset_option('max_rows')
``display.max_colwidth`` sets the maximum width of columns. Cells
of this length or longer will be truncated with an ellipsis.
.. ipython:: python
df = pd.DataFrame(np.array([['foo', 'bar', 'bim', 'uncomfortably long string'],
['horse', 'cow', 'banana', 'apple']]))
pd.set_option('max_colwidth',40)
df
pd.set_option('max_colwidth', 6)
df
pd.reset_option('max_colwidth')
``display.max_info_columns`` sets a threshold for when by-column info
will be given.
.. ipython:: python
df = pd.DataFrame(np.random.randn(10,10))
pd.set_option('max_info_columns', 11)
df.info()
pd.set_option('max_info_columns', 5)
df.info()
pd.reset_option('max_info_columns')
``display.max_info_rows``: ``df.info()`` will usually show null-counts for each column.
For large frames this can be quite slow. ``max_info_rows`` and ``max_info_cols``
limit this null check only to frames with smaller dimensions then specified. Note that you
can specify the option ``df.info(null_counts=True)`` to override on showing a particular frame.
.. ipython:: python
df = pd.DataFrame(np.random.choice([0,1,np.nan], size=(10,10)))
df
pd.set_option('max_info_rows', 11)
df.info()
pd.set_option('max_info_rows', 5)
df.info()
pd.reset_option('max_info_rows')
``display.precision`` sets the output display precision in terms of decimal places.
This is only a suggestion.
.. ipython:: python
df = pd.DataFrame(np.random.randn(5,5))
pd.set_option('precision',7)
df
pd.set_option('precision',4)
df
``display.chop_threshold`` sets at what level pandas rounds to zero when
it displays a Series of DataFrame. This setting does not change the
precision at which the number is stored.
.. ipython:: python
df = pd.DataFrame(np.random.randn(6,6))
pd.set_option('chop_threshold', 0)
df
pd.set_option('chop_threshold', .5)
df
pd.reset_option('chop_threshold')
``display.colheader_justify`` controls the justification of the headers.
The options are 'right', and 'left'.
.. ipython:: python
df = pd.DataFrame(np.array([np.random.randn(6), np.random.randint(1,9,6)*.1, np.zeros(6)]).T,
columns=['A', 'B', 'C'], dtype='float')
pd.set_option('colheader_justify', 'right')
df
pd.set_option('colheader_justify', 'left')
df
pd.reset_option('colheader_justify')
.. _options.available:
Available Options
-----------------
======================================= ============ ==================================
Option Default Function
======================================= ============ ==================================
display.chop_threshold None If set to a float value, all float
values smaller then the given
threshold will be displayed as
exactly 0 by repr and friends.
display.colheader_justify right Controls the justification of
column headers. used by DataFrameFormatter.
display.column_space 12 No description available.
display.date_dayfirst False When True, prints and parses dates
with the day first, eg 20/01/2005
display.date_yearfirst False When True, prints and parses dates
with the year first, eg 2005/01/20
display.encoding UTF-8 Defaults to the detected encoding
of the console. Specifies the encoding
to be used for strings returned by
to_string, these are generally strings
meant to be displayed on the console.
display.expand_frame_repr True Whether to print out the full DataFrame
repr for wide DataFrames across
multiple lines, `max_columns` is
still respected, but the output will
wrap-around across multiple "pages"
if its width exceeds `display.width`.
display.float_format None The callable should accept a floating
point number and return a string with
the desired format of the number.
This is used in some places like
SeriesFormatter.
See core.format.EngFormatter for an example.
display.large_repr truncate For DataFrames exceeding max_rows/max_cols,
the repr (and HTML repr) can show
a truncated table (the default),
or switch to the view from df.info()
(the behaviour in earlier versions of pandas).
allowable settings, ['truncate', 'info']
display.latex.repr False Whether to produce a latex DataFrame
representation for jupyter frontends
that support it.
display.latex.escape True Escapes special characters in DataFrames, when
using the to_latex method.
display.latex.longtable False Specifies if the to_latex method of a DataFrame
uses the longtable format.
display.latex.multicolumn True Combines columns when using a MultiIndex
display.latex.multicolumn_format 'l' Alignment of multicolumn labels
display.latex.multirow False Combines rows when using a MultiIndex.
Centered instead of top-aligned,
separated by clines.
display.max_columns 0 or 20 max_rows and max_columns are used
in __repr__() methods to decide if
to_string() or info() is used to
render an object to a string. In
case Python/IPython is running in
a terminal this is set to 0 by default and
pandas will correctly auto-detect
the width of the terminal and switch to
a smaller format in case all columns
would not fit vertically. The IPython
notebook, IPython qtconsole, or IDLE
do not run in a terminal and hence
it is not possible to do correct
auto-detection, in which case the default
is set to 20. 'None' value means unlimited.
display.max_colwidth 50 The maximum width in characters of
a column in the repr of a pandas
data structure. When the column overflows,
a "..." placeholder is embedded in
the output.
display.max_info_columns 100 max_info_columns is used in DataFrame.info
method to decide if per column information
will be printed.
display.max_info_rows 1690785 df.info() will usually show null-counts
for each column. For large frames
this can be quite slow. max_info_rows
and max_info_cols limit this null
check only to frames with smaller
dimensions then specified.
display.max_rows 60 This sets the maximum number of rows
pandas should output when printing
out various output. For example,
this value determines whether the
repr() for a dataframe prints out
fully or just a summary repr.
'None' value means unlimited.
display.max_seq_items 100 when pretty-printing a long sequence,
no more then `max_seq_items` will
be printed. If items are omitted,
they will be denoted by the addition
of "..." to the resulting string.
If set to None, the number of items
to be printed is unlimited.
display.memory_usage True This specifies if the memory usage of
a DataFrame should be displayed when the
df.info() method is invoked.
display.multi_sparse True "Sparsify" MultiIndex display (don't
display repeated elements in outer
levels within groups)
display.notebook_repr_html True When True, IPython notebook will
use html representation for
pandas objects (if it is available).
display.pprint_nest_depth 3 Controls the number of nested levels
to process when pretty-printing
display.precision 6 Floating point output precision in
terms of number of places after the
decimal, for regular formatting as well
as scientific notation. Similar to
numpy's ``precision`` print option
display.show_dimensions truncate Whether to print out dimensions
at the end of DataFrame repr.
If 'truncate' is specified, only
print out the dimensions if the
frame is truncated (e.g. not display
all rows and/or columns)
display.width 80 Width of the display in characters.
In case python/IPython is running in
a terminal this can be set to None
and pandas will correctly auto-detect
the width. Note that the IPython notebook,
IPython qtconsole, or IDLE do not run in a
terminal and hence it is not possible
to correctly detect the width.
display.html.table_schema False Whether to publish a Table Schema
representation for frontends that
support it.
display.html.border 1 A ``border=value`` attribute is
inserted in the ``<table>`` tag
for the DataFrame HTML repr.
display.html.use_mathjax True When True, Jupyter notebook will process
table contents using MathJax, rendering
mathematical expressions enclosed by the
dollar symbol.
io.excel.xls.writer xlwt The default Excel writer engine for
'xls' files.
io.excel.xlsm.writer openpyxl The default Excel writer engine for
'xlsm' files. Available options:
'openpyxl' (the default).
io.excel.xlsx.writer openpyxl The default Excel writer engine for
'xlsx' files.
io.hdf.default_format None default format writing format, if
None, then put will default to
'fixed' and append will default to
'table'
io.hdf.dropna_table True drop ALL nan rows when appending
to a table
io.parquet.engine None The engine to use as a default for
parquet reading and writing. If None
then try 'pyarrow' and 'fastparquet'
mode.chained_assignment warn Controls ``SettingWithCopyWarning``:
'raise', 'warn', or None. Raise an
exception, warn, or no action if
trying to use :ref:`chained assignment <indexing.evaluation_order>`.
mode.sim_interactive False Whether to simulate interactive mode
for purposes of testing.
mode.use_inf_as_na False True means treat None, NaN, -INF,
INF as NA (old way), False means
None and NaN are null, but INF, -INF
are not NA (new way).
compute.use_bottleneck True Use the bottleneck library to accelerate
computation if it is installed.
compute.use_numexpr True Use the numexpr library to accelerate
computation if it is installed.
plotting.matplotlib.register_converters True Register custom converters with
matplotlib. Set to False to de-register.
======================================= ============ ==================================
.. _basics.console_output:
Number Formatting
------------------
pandas also allows you to set how numbers are displayed in the console.
This option is not set through the ``set_options`` API.
Use the ``set_eng_float_format`` function
to alter the floating-point formatting of pandas objects to produce a particular
format.
For instance:
.. ipython:: python
import numpy as np
pd.set_eng_float_format(accuracy=3, use_eng_prefix=True)
s = pd.Series(np.random.randn(5), index=['a', 'b', 'c', 'd', 'e'])
s/1.e3
s/1.e6
.. ipython:: python
:suppress:
:okwarning:
pd.reset_option('^display\.')
To round floats on a case-by-case basis, you can also use :meth:`~pandas.Series.round` and :meth:`~pandas.DataFrame.round`.
.. _options.east_asian_width:
Unicode Formatting
------------------
.. warning::
Enabling this option will affect the performance for printing of DataFrame and Series (about 2 times slower).
Use only when it is actually required.
Some East Asian countries use Unicode characters whose width corresponds to two Latin characters.
If a DataFrame or Series contains these characters, the default output mode may not align them properly.
.. note:: Screen captures are attached for each output to show the actual results.
.. ipython:: python
df = pd.DataFrame({u'国籍': ['UK', u'日本'], u'名前': ['Alice', u'しのぶ']})
df;
.. image:: _static/option_unicode01.png
Enabling ``display.unicode.east_asian_width`` allows pandas to check each character's "East Asian Width" property.
These characters can be aligned properly by setting this option to ``True``. However, this will result in longer render
times than the standard ``len`` function.
.. ipython:: python
pd.set_option('display.unicode.east_asian_width', True)
df;
.. image:: _static/option_unicode02.png
In addition, Unicode characters whose width is "Ambiguous" can either be 1 or 2 characters wide depending on the
terminal setting or encoding. The option ``display.unicode.ambiguous_as_wide`` can be used to handle the ambiguity.
By default, an "Ambiguous" character's width, such as "¡" (inverted exclamation) in the example below, is taken to be 1.
.. ipython:: python
df = pd.DataFrame({'a': ['xxx', u'¡¡'], 'b': ['yyy', u'¡¡']})
df;
.. image:: _static/option_unicode03.png
Enabling ``display.unicode.ambiguous_as_wide`` makes pandas interpret these characters' widths to be 2.
(Note that this option will only be effective when ``display.unicode.east_asian_width`` is enabled.)
However, setting this option incorrectly for your terminal will cause these characters to be aligned incorrectly:
.. ipython:: python
pd.set_option('display.unicode.ambiguous_as_wide', True)
df;
.. image:: _static/option_unicode04.png
.. ipython:: python
:suppress:
pd.set_option('display.unicode.east_asian_width', False)
pd.set_option('display.unicode.ambiguous_as_wide', False)
.. _options.table_schema:
Table Schema Display
--------------------
.. versionadded:: 0.20.0
``DataFrame`` and ``Series`` will publish a Table Schema representation
by default. False by default, this can be enabled globally with the
``display.html.table_schema`` option:
.. ipython:: python
pd.set_option('display.html.table_schema', True)
Only ``'display.max_rows'`` are serialized and published.
.. ipython:: python
:suppress:
pd.reset_option('display.html.table_schema')
|
