1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
|
.. -*- rst -*-
.. highlightlang:: none
.. groonga-command
.. database: commands_normalize
``normalize``
=============
.. note::
This command is an experimental feature.
This command may be changed in the future.
Summary
-------
``normalize`` command normalizes text by the specified normalizer.
There is no need to create table to use ``normalize`` command.
It is useful for you to check the results of normalizer.
Syntax
--------
This command takes three parameters.
``normalizer`` and ``string`` are required. Others are optional::
normalize normalizer
string
[flags=NONE]
Usage
-----
Here is a simple example of ``normalize`` command.
.. groonga-command
.. include:: ../../example/reference/commands/normalize/normalizer_auto_ascii.log
.. normalize NormalizerAuto "aBcDe 123"
Parameters
----------
This section describes parameters of ``normalizer``.
Required parameters
^^^^^^^^^^^^^^^^^^^
There are required parameters, ``normalizer`` and ``string``.
``normalizer``
""""""""""""""
Specifies the normalizer name. ``normalize`` command uses the
normalizer that is named ``normalizer``.
See :doc:`/reference/normalizers` about built-in normalizers.
Here is an example to use built-in ``NormalizerAuto`` normalizer.
TODO
If you want to use other normalizers, you need to register additional
normalizer plugin by :doc:`register` command. For example, you can use
MySQL compatible normalizer by registering `groonga-normalizer-mysql
<https://github.com/groonga/groonga-normalizer-mysql>`_.
``string``
""""""""""
Specifies any string which you want to normalize.
If you want to include spaces in ``string``, you need to quote
``string`` by single quotation (``'``) or double quotation (``"``).
Here is an example to use spaces in ``string``.
TODO
Optional parameters
^^^^^^^^^^^^^^^^^^^
There are optional parameters.
``flags``
"""""""""
Specifies a normalization customize options. You can specify
multiple options separated by "``|``". For example,
``REMOVE_BLANK|WITH_TYPES``.
Here are available flags.
.. list-table::
:header-rows: 1
* - Flag
- Description
* - ``NONE``
- Just ignored.
* - ``REMOVE_BLANK``
- TODO
* - ``WITH_TYPES``
- TODO
* - ``WITH_CHECKS``
- TODO
* - ``REMOVE_TOKENIZED_DELIMITER``
- TODO
Here is an example that uses ``REMOVE_BLANK``.
TODO
Here is an example that uses ``WITH_TYPES``.
TODO
Here is an example that uses ``REMOVE_TOKENIZED_DELIMITER``.
TODO
Return value
------------
::
[HEADER, normalized_text]
``HEADER``
See :doc:`/reference/command/output_format` about ``HEADER``.
``normalized_text``
``normalized_text`` is an object that has the following attributes.
.. list-table::
:header-rows: 1
* - Name
- Description
* - ``normalized``
- The normalized text.
* - ``types``
- An array of types of the normalized text. The N-th ``types`` shows
the type of the N-th character in ``normalized``.
See also
--------
* :doc:`/reference/normalizers`
|
