File: data_utils.rst

package info (click to toggle)
pytorch-text 0.14.1-2
  • links: PTS, VCS
  • area: main
  • in suites: bookworm
  • size: 11,560 kB
  • sloc: python: 14,197; cpp: 2,404; sh: 214; makefile: 20
file content (18 lines) | stat: -rw-r--r-- 344 bytes parent folder | download 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
 
.. role:: hidden
    :class: hidden-section

torchtext.data.utils
===========================

.. automodule:: torchtext.data.utils
.. currentmodule:: torchtext.data.utils

:hidden:`get_tokenizer`
~~~~~~~~~~~~~~~~~~~~~~~~~

.. autofunction:: get_tokenizer

:hidden:`ngrams_iterator`
~~~~~~~~~~~~~~~~~~~~~~~~~

.. autofunction:: ngrams_iterator