1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
|
=head1 NAME
HTML::FormFu::Manual::Unicode - Working with unicode
=head1 DESCRIPTION
Working with unicode.
For a practical example, see the Catalyst application in the
C<examples/unicode> directory in this distribution.
=head1 ASSUMPTIONS
In this tutorial, we're assuming that all encodings are UTF-8. It's
relatively simple to combine different encodings from different sources,
but that's beyond the scope of this tutorial.
For simplicity, we're also going to assume that you're using L<Catalyst>
for your web-framework, L<DBIx::Class> for your database ORM,
L<TT|Template> for your templating system, and YAML format C<HTML::FormFu>
configuration files, with L<YAML::XS> installed. However, the principles
we'll cover should translate to whatever technologies you chose to work with.
=head1 BASICS
To make it short and sweet: you must decode all data going into your
program, and encode all data coming from your program.
Skip to L</CHANGES REQUIRED> if you want to see what you need to do without
any other explanation.
=head1 INPUT
=head2 Input parameters from the browser
If you're using C<Catalyst>, L<Catalyst::Plugin::Unicode> will decode all
input parameters sent from the browser to your application - see
L</Catalyst Configuration>.
If you're using some other framework or, in any case, you need to decode
the input parameters yourself, please take a look at
L<HTML::FormFu::Filter::Encode>.
=head2 Data from the database
If you're using L<DBIx::Class>, L<DBIx::Class::UTF8Columns> is likely the
best options, as it will decode all input retrieved from the database -
see L</DBIx::Class Configuration>.
In other cases (i.e. plain DBI), you still need to decode the string data
coming from the database. This varies depending on the database server.
For MySQL, for instance, you can use the C<mysql_enable_utf8> attribute:
see L<DBD::mysql> documentation for details.
=head2 Your template files
Set TT to decode all template files - see L</TT Configuration>.
=head2 HTML::FormFu's own template files
Set C<HTML::FormFu> to decode all template files - see
L</HTML::FormFu Template Configuration>.
=head2 HTML::FormFu form configuration files
If you're using C<YAML> config files, your files will automatically be
decoded by C<load_config_file|HTML::FormFu/load_config_file> and
C<load_config_filestem|HTML::FormFu/load_config_filestem>.
If you have L<Config::General> config files, your files will automatically
be decoded by C<load_config_file|HTML::FormFu/load_config_file> and
C<load_config_filestem|HTML::FormFu/load_config_filestem>, which
automatically sets L<Config::General's|Config::General> C<-UTF8> setting.
=head2 Your perl source code
Any perl source files which contain Unicode characters must use the
L<utf8> module.
=head1 OUTPUT
=head2 Data saved to the database
With C<DBIx::Class>, L<DBIx::Class::UTF8Columns> will encode all data sent
to the database - see L</DBIx::Class Configuration>.
=head2 HTML sent to the browser
With C<Catalyst>, L<Catalyst::Plugin::Unicode> will encode all output sent
from your application to the browser - see L</Catalyst Configuration>.
In other circumstances you need to be sure to output your Unicode (decoded)
strings in UTF-8. To do this you can encode your output before it's sent
to the browser with something like:
use utf8;
if ( $output && utf8::is_utf8($output) ){
utf8::encode( $output ); # Encodes in-place
}
Another option is to set the C<binmode> for C<STDOUT>:
bindmode STDOUT, ':utf8';
However, be sure to do this B<only> when sending UTF-8 data: if you're
serving images, PFD files, etc, C<binmode> should remain set to C<:raw>.
=head1 CHANGES REQUIRED
=head2 Catalyst Configuration
Add L<Catalyst::Plugin::Unicode> to the list of Catalyst plugins:
use Catalyst qw( ConfigLoader Static::Simple Unicode );
=head2 DBIx::Class Configuration
Add L<DBIx::Class::UTF8Columns> to the list of components loaded, for each
table that has columns storing unicode:
__PACKAGE__->load_components( qw( UTF8Columns HTML::FormFu PK::Auto Core ) );
Pass each column name that will store unicode to C<utf8_columns()>:
__PACKAGE__->utf8_columns( qw( lastname firstname ) );
=head2 TT Configuration
Tell TT to decode all template files, by adding the following to your
application config in MyApp.pm
package MyApp;
use strict;
use parent 'Catalyst';
use Catalyst qw( ConfigLoader );
MyApp->config({
'View::TT' => {
ENCODING => 'UTF-8',
},
});
1;
=head2 HTML::FormFu Template Configuration
Make C<HTML::FormFu> tell TT to decode all template files, by adding the
following to your C<myapp.yml> Catalyst configuration file:
package MyApp;
use strict;
use parent 'Catalyst';
use Catalyst qw( ConfigLoader );
MyApp->config({
'Controller::HTML::FormFu' => {
constructor => {
tt_args => {
ENCODING => 'UTF-8',
},
},
},
});
1;
These above 2 examples should be combined, like so:
package MyApp;
use strict;
use parent 'Catalyst';
use Catalyst qw( ConfigLoader );
MyApp->config({
'Controller::HTML::FormFu' => {
constructor => {
tt_args => {
ENCODING => 'UTF-8',
},
},
},
'View::TT' => {
ENCODING => 'UTF-8',
},
});
1;
=head1 AUTHORS
Carl Franks C<cfranks@cpan.org>
Michele Beltrame C<arthas@cpan.org> (contributions)
=head1 COPYRIGHT
This document is free, you can redistribute it and/or modify it
under the same terms as Perl itself.
=cut
|
