File: timedeltas.rst

package info (click to toggle)
pandas 1.5.3%2Bdfsg-2
  • links: PTS, VCS
  • area: main
  • in suites: bookworm
  • size: 56,516 kB
  • sloc: python: 382,477; ansic: 8,695; sh: 119; xml: 102; makefile: 97
file content (491 lines) | stat: -rw-r--r-- 12,913 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
.. _timedeltas:

{{ header }}

.. _timedeltas.timedeltas:

***********
Time deltas
***********

Timedeltas are differences in times, expressed in difference units, e.g. days, hours, minutes,
seconds. They can be both positive and negative.

``Timedelta`` is a subclass of ``datetime.timedelta``, and behaves in a similar manner,
but allows compatibility with ``np.timedelta64`` types as well as a host of custom representation,
parsing, and attributes.

Parsing
-------

You can construct a ``Timedelta`` scalar through various arguments, including `ISO 8601 Duration`_ strings.

.. ipython:: python

   import datetime

   # strings
   pd.Timedelta("1 days")
   pd.Timedelta("1 days 00:00:00")
   pd.Timedelta("1 days 2 hours")
   pd.Timedelta("-1 days 2 min 3us")

   # like datetime.timedelta
   # note: these MUST be specified as keyword arguments
   pd.Timedelta(days=1, seconds=1)

   # integers with a unit
   pd.Timedelta(1, unit="d")

   # from a datetime.timedelta/np.timedelta64
   pd.Timedelta(datetime.timedelta(days=1, seconds=1))
   pd.Timedelta(np.timedelta64(1, "ms"))

   # negative Timedeltas have this string repr
   # to be more consistent with datetime.timedelta conventions
   pd.Timedelta("-1us")

   # a NaT
   pd.Timedelta("nan")
   pd.Timedelta("nat")

   # ISO 8601 Duration strings
   pd.Timedelta("P0DT0H1M0S")
   pd.Timedelta("P0DT0H0M0.000000123S")

:ref:`DateOffsets<timeseries.offsets>` (``Day, Hour, Minute, Second, Milli, Micro, Nano``) can also be used in construction.

.. ipython:: python

   pd.Timedelta(pd.offsets.Second(2))

Further, operations among the scalars yield another scalar ``Timedelta``.

.. ipython:: python

   pd.Timedelta(pd.offsets.Day(2)) + pd.Timedelta(pd.offsets.Second(2)) + pd.Timedelta(
       "00:00:00.000123"
   )

to_timedelta
~~~~~~~~~~~~

Using the top-level ``pd.to_timedelta``, you can convert a scalar, array, list,
or Series from a recognized timedelta format / value into a ``Timedelta`` type.
It will construct Series if the input is a Series, a scalar if the input is
scalar-like, otherwise it will output a ``TimedeltaIndex``.

You can parse a single string to a Timedelta:

.. ipython:: python

   pd.to_timedelta("1 days 06:05:01.00003")
   pd.to_timedelta("15.5us")

or a list/array of strings:

.. ipython:: python

   pd.to_timedelta(["1 days 06:05:01.00003", "15.5us", "nan"])

The ``unit`` keyword argument specifies the unit of the Timedelta if the input
is numeric:

.. ipython:: python

   pd.to_timedelta(np.arange(5), unit="s")
   pd.to_timedelta(np.arange(5), unit="d")

.. warning::
    If a string or array of strings is passed as an input then the ``unit`` keyword
    argument will be ignored. If a string without units is passed then the default
    unit of nanoseconds is assumed.

.. _timedeltas.limitations:

Timedelta limitations
~~~~~~~~~~~~~~~~~~~~~

pandas represents ``Timedeltas`` in nanosecond resolution using
64 bit integers. As such, the 64 bit integer limits determine
the ``Timedelta`` limits.

.. ipython:: python

   pd.Timedelta.min
   pd.Timedelta.max

.. _timedeltas.operations:

Operations
----------

You can operate on Series/DataFrames and construct ``timedelta64[ns]`` Series through
subtraction operations on ``datetime64[ns]`` Series, or ``Timestamps``.

.. ipython:: python

   s = pd.Series(pd.date_range("2012-1-1", periods=3, freq="D"))
   td = pd.Series([pd.Timedelta(days=i) for i in range(3)])
   df = pd.DataFrame({"A": s, "B": td})
   df
   df["C"] = df["A"] + df["B"]
   df
   df.dtypes

   s - s.max()
   s - datetime.datetime(2011, 1, 1, 3, 5)
   s + datetime.timedelta(minutes=5)
   s + pd.offsets.Minute(5)
   s + pd.offsets.Minute(5) + pd.offsets.Milli(5)

Operations with scalars from a ``timedelta64[ns]`` series:

.. ipython:: python

   y = s - s[0]
   y

Series of timedeltas with ``NaT`` values are supported:

.. ipython:: python

   y = s - s.shift()
   y

Elements can be set to ``NaT`` using ``np.nan`` analogously to datetimes:

.. ipython:: python

   y[1] = np.nan
   y

Operands can also appear in a reversed order (a singular object operated with a Series):

.. ipython:: python

   s.max() - s
   datetime.datetime(2011, 1, 1, 3, 5) - s
   datetime.timedelta(minutes=5) + s

``min, max`` and the corresponding ``idxmin, idxmax`` operations are supported on frames:

.. ipython:: python

   A = s - pd.Timestamp("20120101") - pd.Timedelta("00:05:05")
   B = s - pd.Series(pd.date_range("2012-1-2", periods=3, freq="D"))

   df = pd.DataFrame({"A": A, "B": B})
   df

   df.min()
   df.min(axis=1)

   df.idxmin()
   df.idxmax()

``min, max, idxmin, idxmax`` operations are supported on Series as well. A scalar result will be a ``Timedelta``.

.. ipython:: python

   df.min().max()
   df.min(axis=1).min()

   df.min().idxmax()
   df.min(axis=1).idxmin()

You can fillna on timedeltas, passing a timedelta to get a particular value.

.. ipython:: python

   y.fillna(pd.Timedelta(0))
   y.fillna(pd.Timedelta(10, unit="s"))
   y.fillna(pd.Timedelta("-1 days, 00:00:05"))

You can also negate, multiply and use ``abs`` on ``Timedeltas``:

.. ipython:: python

   td1 = pd.Timedelta("-1 days 2 hours 3 seconds")
   td1
   -1 * td1
   -td1
   abs(td1)

.. _timedeltas.timedeltas_reductions:

Reductions
----------

Numeric reduction operation for ``timedelta64[ns]`` will return ``Timedelta`` objects. As usual
``NaT`` are skipped during evaluation.

.. ipython:: python

   y2 = pd.Series(
       pd.to_timedelta(["-1 days +00:00:05", "nat", "-1 days +00:00:05", "1 days"])
   )
   y2
   y2.mean()
   y2.median()
   y2.quantile(0.1)
   y2.sum()

.. _timedeltas.timedeltas_convert:

Frequency conversion
--------------------

Timedelta Series, ``TimedeltaIndex``, and ``Timedelta`` scalars can be converted to other 'frequencies' by dividing by another timedelta,
or by astyping to a specific timedelta type. These operations yield Series and propagate ``NaT`` -> ``nan``.
Note that division by the NumPy scalar is true division, while astyping is equivalent of floor division.

.. ipython:: python

   december = pd.Series(pd.date_range("20121201", periods=4))
   january = pd.Series(pd.date_range("20130101", periods=4))
   td = january - december

   td[2] += datetime.timedelta(minutes=5, seconds=3)
   td[3] = np.nan
   td

   # to days
   td / np.timedelta64(1, "D")
   td.astype("timedelta64[D]")

   # to seconds
   td / np.timedelta64(1, "s")
   td.astype("timedelta64[s]")

   # to months (these are constant months)
   td / np.timedelta64(1, "M")

Dividing or multiplying a ``timedelta64[ns]`` Series by an integer or integer Series
yields another ``timedelta64[ns]`` dtypes Series.

.. ipython:: python

   td * -1
   td * pd.Series([1, 2, 3, 4])

Rounded division (floor-division) of a ``timedelta64[ns]`` Series by a scalar
``Timedelta`` gives a series of integers.

.. ipython:: python

   td // pd.Timedelta(days=3, hours=4)
   pd.Timedelta(days=3, hours=4) // td

.. _timedeltas.mod_divmod:

The mod (%) and divmod operations are defined for ``Timedelta`` when operating with another timedelta-like or with a numeric argument.

.. ipython:: python

   pd.Timedelta(hours=37) % datetime.timedelta(hours=2)

   # divmod against a timedelta-like returns a pair (int, Timedelta)
   divmod(datetime.timedelta(hours=2), pd.Timedelta(minutes=11))

   # divmod against a numeric returns a pair (Timedelta, Timedelta)
   divmod(pd.Timedelta(hours=25), 86400000000000)

Attributes
----------

You can access various components of the ``Timedelta`` or ``TimedeltaIndex`` directly using the attributes ``days,seconds,microseconds,nanoseconds``. These are identical to the values returned by ``datetime.timedelta``, in that, for example, the ``.seconds`` attribute represents the number of seconds >= 0 and < 1 day. These are signed according to whether the ``Timedelta`` is signed.

These operations can also be directly accessed via the ``.dt`` property of the ``Series`` as well.

.. note::

   Note that the attributes are NOT the displayed values of the ``Timedelta``. Use ``.components`` to retrieve the displayed values.

For a ``Series``:

.. ipython:: python

   td.dt.days
   td.dt.seconds

You can access the value of the fields for a scalar ``Timedelta`` directly.

.. ipython:: python

   tds = pd.Timedelta("31 days 5 min 3 sec")
   tds.days
   tds.seconds
   (-tds).seconds

You can use the ``.components`` property to access a reduced form of the timedelta. This returns a ``DataFrame`` indexed
similarly to the ``Series``. These are the *displayed* values of the ``Timedelta``.

.. ipython:: python

   td.dt.components
   td.dt.components.seconds

.. _timedeltas.isoformat:

You can convert a ``Timedelta`` to an `ISO 8601 Duration`_ string with the
``.isoformat`` method

.. ipython:: python

    pd.Timedelta(
        days=6, minutes=50, seconds=3, milliseconds=10, microseconds=10, nanoseconds=12
    ).isoformat()

.. _ISO 8601 Duration: https://en.wikipedia.org/wiki/ISO_8601#Durations

.. _timedeltas.index:

TimedeltaIndex
--------------

To generate an index with time delta, you can use either the :class:`TimedeltaIndex` or
the :func:`timedelta_range` constructor.

Using ``TimedeltaIndex`` you can pass string-like, ``Timedelta``, ``timedelta``,
or ``np.timedelta64`` objects. Passing ``np.nan/pd.NaT/nat`` will represent missing values.

.. ipython:: python

   pd.TimedeltaIndex(
       [
           "1 days",
           "1 days, 00:00:05",
           np.timedelta64(2, "D"),
           datetime.timedelta(days=2, seconds=2),
       ]
   )

The string 'infer' can be passed in order to set the frequency of the index as the
inferred frequency upon creation:

.. ipython:: python

   pd.TimedeltaIndex(["0 days", "10 days", "20 days"], freq="infer")

Generating ranges of time deltas
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Similar to :func:`date_range`, you can construct regular ranges of a ``TimedeltaIndex``
using :func:`timedelta_range`.  The default frequency for ``timedelta_range`` is
calendar day:

.. ipython:: python

   pd.timedelta_range(start="1 days", periods=5)

Various combinations of ``start``, ``end``, and ``periods`` can be used with
``timedelta_range``:

.. ipython:: python

   pd.timedelta_range(start="1 days", end="5 days")

   pd.timedelta_range(end="10 days", periods=4)

The ``freq`` parameter can passed a variety of :ref:`frequency aliases <timeseries.offset_aliases>`:

.. ipython:: python

   pd.timedelta_range(start="1 days", end="2 days", freq="30T")

   pd.timedelta_range(start="1 days", periods=5, freq="2D5H")


Specifying ``start``, ``end``, and ``periods`` will generate a range of evenly spaced
timedeltas from ``start`` to ``end`` inclusively, with ``periods`` number of elements
in the resulting ``TimedeltaIndex``:

.. ipython:: python

   pd.timedelta_range("0 days", "4 days", periods=5)

   pd.timedelta_range("0 days", "4 days", periods=10)

Using the TimedeltaIndex
~~~~~~~~~~~~~~~~~~~~~~~~

Similarly to other of the datetime-like indices, ``DatetimeIndex`` and ``PeriodIndex``, you can use
``TimedeltaIndex`` as the index of pandas objects.

.. ipython:: python

   s = pd.Series(
       np.arange(100),
       index=pd.timedelta_range("1 days", periods=100, freq="h"),
   )
   s

Selections work similarly, with coercion on string-likes and slices:

.. ipython:: python

   s["1 day":"2 day"]
   s["1 day 01:00:00"]
   s[pd.Timedelta("1 day 1h")]

Furthermore you can use partial string selection and the range will be inferred:

.. ipython:: python

   s["1 day":"1 day 5 hours"]

Operations
~~~~~~~~~~

Finally, the combination of ``TimedeltaIndex`` with ``DatetimeIndex`` allow certain combination operations that are NaT preserving:

.. ipython:: python

   tdi = pd.TimedeltaIndex(["1 days", pd.NaT, "2 days"])
   tdi.to_list()
   dti = pd.date_range("20130101", periods=3)
   dti.to_list()
   (dti + tdi).to_list()
   (dti - tdi).to_list()

Conversions
~~~~~~~~~~~

Similarly to frequency conversion on a ``Series`` above, you can convert these indices to yield another Index.

.. ipython:: python

   tdi / np.timedelta64(1, "s")
   tdi.astype("timedelta64[s]")

Scalars type ops work as well. These can potentially return a *different* type of index.

.. ipython:: python

   # adding or timedelta and date -> datelike
   tdi + pd.Timestamp("20130101")

   # subtraction of a date and a timedelta -> datelike
   # note that trying to subtract a date from a Timedelta will raise an exception
   (pd.Timestamp("20130101") - tdi).to_list()

   # timedelta + timedelta -> timedelta
   tdi + pd.Timedelta("10 days")

   # division can result in a Timedelta if the divisor is an integer
   tdi / 2

   # or a Float64Index if the divisor is a Timedelta
   tdi / tdi[0]

.. _timedeltas.resampling:

Resampling
----------

Similar to :ref:`timeseries resampling <timeseries.resampling>`, we can resample with a ``TimedeltaIndex``.

.. ipython:: python

   s.resample("D").mean()