1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
|
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/strict.dtd">
<!--This file was generated by EmPy from README.html.in : do not edit-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<meta name="author" content="John J. Lee <jjl@pobox.com>">
<meta name="date" content="2008-07-19">
<meta name="keywords" content="form,HTML,Python,web,client,client-side">
<title>ClientForm</title>
<style type="text/css" media="screen">@import "../styles/style.css";</style>
</head>
<body>
<div id="sf"><a href="http://sourceforge.net">
<img src="http://sourceforge.net/sflogo.php?group_id=48205&type=2"
width="125" height="37" alt="SourceForge.net Logo"></a></div>
<h1>ClientForm</h1>
<div id="Content">
<p>ClientForm is a Python module for handling HTML forms on the client
side, useful for parsing HTML forms, filling them in and returning the
completed forms to the server. It developed from a port of Gisle Aas'
Perl module <code>HTML::Form</code>, from the <a
href="http://www.linpro.no/lwp/">libwww-perl</a> library, but the
interface is not the same.
<p>Simple working example:
<pre><span class="pykw">from</span> urllib2 <span class="pykw">import</span> urlopen
<span class="pykw">from</span> ClientForm <span class="pykw">import</span> ParseResponse
response = urlopen(<span class="pystr">"http://wwwsearch.sourceforge.net/ClientForm/example.html"</span>)
forms = ParseResponse(response, backwards_compat=False)
form = forms[0]
<span class="pykw">print</span> form
form[<span class="pystr">"comments"</span>] = <span class="pystr">"Thanks, Gisle"</span>
<span class="pycmt"># form.click() returns a urllib2.Request object
</span><span class="pycmt"># (see HTMLForm.click.__doc__ if you don't have urllib2)
</span><span class="pykw">print</span> urlopen(form.click()).read()</pre>
<p>A more complicated working example (<em><strong>Note</strong>: this
example makes use of the ClientForm 0.2 API; refer to the README.html
file in the latest 0.1 release for the corresponding code for that
version.</em>):
<a name="example"></a>
<pre><span class="pykw">import</span> ClientForm
<span class="pykw">import</span> urllib2
request = urllib2.Request(
<span class="pystr">"http://wwwsearch.sourceforge.net/ClientForm/example.html"</span>)
response = urllib2.urlopen(request)
forms = ClientForm.ParseResponse(response, backwards_compat=False)
response.close()
<span class="pycmt">## f = open("example.html")
</span><span class="pycmt">## forms = ClientForm.ParseFile(f, "http://example.com/example.html",
</span><span class="pycmt">## backwards_compat=False)
</span><span class="pycmt">## f.close()
</span>form = forms[0]
<span class="pykw">print</span> form <span class="pycmt"># very useful!</span>
<span class="pycmt"># A 'control' is a graphical HTML form widget: a text entry box, a
</span><span class="pycmt"># dropdown 'select' list, a checkbox, etc.
</span>
<span class="pycmt"># Indexing allows setting and retrieval of control values
</span>original_text = form[<span class="pystr">"comments"</span>] <span class="pycmt"># a string, NOT a Control instance</span>
form[<span class="pystr">"comments"</span>] = <span class="pystr">"Blah."</span>
<span class="pycmt"># Controls that represent lists (checkbox, select and radio lists) are
</span><span class="pycmt"># ListControl instances. Their values are sequences of list item names.
</span><span class="pycmt"># They come in two flavours: single- and multiple-selection:
</span>form[<span class="pystr">"favorite_cheese"</span>] = [<span class="pystr">"brie"</span>] <span class="pycmt"># single</span>
form[<span class="pystr">"cheeses"</span>] = [<span class="pystr">"parmesan"</span>, <span class="pystr">"leicester"</span>, <span class="pystr">"cheddar"</span>] <span class="pycmt"># multi</span>
<span class="pycmt"># equivalent, but more flexible:
</span>form.set_value([<span class="pystr">"parmesan"</span>, <span class="pystr">"leicester"</span>, <span class="pystr">"cheddar"</span>], name=<span class="pystr">"cheeses"</span>)
<span class="pycmt"># Add files to FILE controls with .add_file(). Only call this multiple
</span><span class="pycmt"># times if the server is expecting multiple files.
</span><span class="pycmt"># add a file, default value for MIME type, no filename sent to server
</span>form.add_file(open(<span class="pystr">"data.dat"</span>))
<span class="pycmt"># add a second file, explicitly giving MIME type, and telling the server
</span><span class="pycmt"># what the filename is
</span>form.add_file(open(<span class="pystr">"data.txt"</span>), <span class="pystr">"text/plain"</span>, <span class="pystr">"data.txt"</span>)
<span class="pycmt"># All Controls may be disabled (equivalent of greyed-out in browser)...
</span>control = form.find_control(<span class="pystr">"comments"</span>)
<span class="pykw">print</span> control.disabled
<span class="pycmt"># ...or readonly
</span><span class="pykw">print</span> control.readonly
<span class="pycmt"># readonly and disabled attributes can be assigned to
</span>control.disabled = False
<span class="pycmt"># convenience method, used here to make all controls writable (unless
</span><span class="pycmt"># they're disabled):
</span>form.set_all_readonly(False)
<span class="pycmt"># A couple of notes about list controls and HTML:
</span>
<span class="pycmt"># 1. List controls correspond to either a single SELECT element, or
</span><span class="pycmt"># multiple INPUT elements. Items correspond to either OPTION or INPUT
</span><span class="pycmt"># elements. For example, this is a SELECT control, named "control1":
</span>
<span class="pycmt"># <select name="control1">
</span><span class="pycmt"># <option>foo</option>
</span><span class="pycmt"># <option value="1">bar</option>
</span><span class="pycmt"># </select>
</span>
<span class="pycmt"># and this is a CHECKBOX control, named "control2":
</span>
<span class="pycmt"># <input type="checkbox" name="control2" value="foo" id="cbe1">
</span><span class="pycmt"># <input type="checkbox" name="control2" value="bar" id="cbe2">
</span>
<span class="pycmt"># You know the latter is a single control because all the name attributes
</span><span class="pycmt"># are the same.
</span>
<span class="pycmt"># 2. Item names are the strings that go to make up the value that should
</span><span class="pycmt"># be returned to the server. These strings come from various different
</span><span class="pycmt"># pieces of text in the HTML. The HTML standard and the ClientForm
</span><span class="pycmt"># docstrings explain in detail, but playing around with an HTML file,
</span><span class="pycmt"># ParseFile() and 'print form' is very useful to understand this!
</span>
<span class="pycmt"># You can get the Control instances from inside the form...
</span>control = form.find_control(<span class="pystr">"cheeses"</span>, type=<span class="pystr">"select"</span>)
<span class="pykw">print</span> control.name, control.value, control.type
control.value = [<span class="pystr">"mascarpone"</span>, <span class="pystr">"curd"</span>]
<span class="pycmt"># ...and the Item instances from inside the Control
</span>item = control.get(<span class="pystr">"curd"</span>)
<span class="pykw">print</span> item.name, item.selected, item.id, item.attrs
item.selected = False
<span class="pycmt"># Controls may be referred to by label:
</span><span class="pycmt"># find control with label that has a *substring* "Cheeses"
</span><span class="pycmt"># (eg., a label "Please select a cheese" would match).
</span>control = form.find_control(label=<span class="pystr">"select a cheese"</span>)
<span class="pycmt"># You can explicitly say that you're referring to a ListControl:
</span><span class="pycmt"># set value of "cheeses" ListControl
</span>form.set_value([<span class="pystr">"gouda"</span>], name=<span class="pystr">"cheeses"</span>, kind=<span class="pystr">"list"</span>)
<span class="pycmt"># equivalent:
</span>form.find_control(name=<span class="pystr">"cheeses"</span>, kind=<span class="pystr">"list"</span>).value = [<span class="pystr">"gouda"</span>]
<span class="pycmt"># the first example is also almost equivalent to the following (but
</span><span class="pycmt"># insists that the control be a ListControl -- so it will skip any
</span><span class="pycmt"># non-list controls that come before the control we want)
</span>form[<span class="pystr">"cheeses"</span>] = [<span class="pystr">"gouda"</span>]
<span class="pycmt"># The kind argument can also take values "multilist", "singlelist", "text",
</span><span class="pycmt"># "clickable" and "file":
</span><span class="pycmt"># find first control that will accept text, and scribble in it
</span>form.set_value(<span class="pystr">"rhubarb rhubarb"</span>, kind=<span class="pystr">"text"</span>, nr=0)
<span class="pycmt"># find, and set the value of, the first single-selection list control
</span>form.set_value([<span class="pystr">"spam"</span>], kind=<span class="pystr">"singlelist"</span>, nr=0)
<span class="pycmt"># You can find controls with a general predicate function:
</span><span class="pykw">def</span> control_has_caerphilly(control):
<span class="pykw">for</span> item <span class="pykw">in</span> control.items:
<span class="pykw">if</span> item.name == <span class="pystr">"caerphilly"</span>: <span class="pykw">return</span> True
form.find_control(kind=<span class="pystr">"list"</span>, predicate=control_has_caerphilly)
<span class="pycmt"># HTMLForm.controls is a list of all controls in the form
</span><span class="pykw">for</span> control <span class="pykw">in</span> form.controls:
<span class="pykw">if</span> control.value == <span class="pystr">"inquisition"</span>: sys.exit()
<span class="pycmt"># Control.items is a list of all Item instances in the control
</span><span class="pykw">for</span> item <span class="pykw">in</span> form.find_control(<span class="pystr">"cheeses"</span>).items:
<span class="pykw">print</span> item.name
<span class="pycmt"># To remove items from a list control, remove it from .items:
</span>cheeses = form.find_control(<span class="pystr">"cheeses"</span>)
curd = cheeses.get(<span class="pystr">"curd"</span>)
<span class="pykw">del</span> cheeses.items[cheeses.items.index(curd)]
<span class="pycmt"># To add items to a list container, instantiate an Item with its control
</span><span class="pycmt"># and attributes:
</span><span class="pycmt"># Note that you are responsible for getting the attributes correct here,
</span><span class="pycmt"># and these are not quite identical to the original HTML, due to
</span><span class="pycmt"># defaulting rules and a few special attributes (e.g. Items that represent
</span><span class="pycmt"># OPTIONs have a special "contents" key in their .attrs dict). In future
</span><span class="pycmt"># there will be an explicitly supported way of using the parsing logic to
</span><span class="pycmt"># add items and controls from HTML strings without knowing these details.
</span>ClientForm.Item(cheeses, {<span class="pystr">"contents"</span>: <span class="pystr">"mascarpone"</span>,
<span class="pystr">"value"</span>: <span class="pystr">"mascarpone"</span>})
<span class="pycmt"># You can specify list items by label using set/get_value_by_label() and
</span><span class="pycmt"># the label argument of the .get() method. Sometimes labels are easier to
</span><span class="pycmt"># maintain than names, sometimes the other way around.
</span>form.set_value_by_label([<span class="pystr">"Mozzarella"</span>, <span class="pystr">"Caerphilly"</span>], <span class="pystr">"cheeses"</span>)
<span class="pycmt"># Which items are present, selected, and successful?
</span><span class="pycmt"># is the "parmesan" item of the "cheeses" control successful (selected
</span><span class="pycmt"># and not disabled)?
</span><span class="pykw">print</span> <span class="pystr">"parmesan"</span> <span class="pykw">in</span> form[<span class="pystr">"cheeses"</span>]
<span class="pycmt"># is the "parmesan" item of the "cheeses" control selected?
</span><span class="pykw">print</span> <span class="pystr">"parmesan"</span> <span class="pykw">in</span> [
item.name <span class="pykw">for</span> item <span class="pykw">in</span> form.find_control(<span class="pystr">"cheeses"</span>).items <span class="pykw">if</span> item.selected]
<span class="pycmt"># does cheeses control have a "caerphilly" item?
</span><span class="pykw">print</span> <span class="pystr">"caerphilly"</span> <span class="pykw">in</span> [item.name <span class="pykw">for</span> item <span class="pykw">in</span> form.find_control(<span class="pystr">"cheeses"</span>).items]
<span class="pycmt"># Sometimes one wants to set or clear individual items in a list, rather
</span><span class="pycmt"># than setting the whole .value:
</span><span class="pycmt"># select the item named "gorgonzola" in the first control named "cheeses"
</span>form.find_control(<span class="pystr">"cheeses"</span>).get(<span class="pystr">"gorgonzola"</span>).selected = True
<span class="pycmt"># You can be more specific:
</span><span class="pycmt"># deselect "edam" in third CHECKBOX control
</span>form.find_control(type=<span class="pystr">"checkbox"</span>, nr=2).get(<span class="pystr">"edam"</span>).selected = False
<span class="pycmt"># deselect item labelled "Mozzarella" in control with id "chz"
</span>form.find_control(id=<span class="pystr">"chz"</span>).get(label=<span class="pystr">"Mozzarella"</span>).selected = False
<span class="pycmt"># Often, a single checkbox (a CHECKBOX control with a single item) is
</span><span class="pycmt"># present. In that case, the name of the single item isn't of much
</span><span class="pycmt"># interest, so it's a good idea to check and uncheck the box without
</span><span class="pycmt"># using the item name:
</span>form.find_control(<span class="pystr">"smelly"</span>).items[0].selected = True <span class="pycmt"># check</span>
form.find_control(<span class="pystr">"smelly"</span>).items[0].selected = False <span class="pycmt"># uncheck</span>
<span class="pycmt"># Items may be disabled (selecting or de-selecting a disabled item is
</span><span class="pycmt"># not allowed):
</span>control = form.find_control(<span class="pystr">"cheeses"</span>)
<span class="pykw">print</span> control.get(<span class="pystr">"emmenthal"</span>).disabled
control.get(<span class="pystr">"emmenthal"</span>).disabled = True
<span class="pycmt"># enable all items in control
</span>control.set_all_items_disabled(False)
request2 = form.click() <span class="pycmt"># urllib2.Request object</span>
<span class="pykw">try</span>:
response2 = urllib2.urlopen(request2)
<span class="pykw">except</span> urllib2.HTTPError, response2:
<span class="pykw">pass</span>
<span class="pykw">print</span> response2.geturl()
<span class="pykw">print</span> response2.info() <span class="pycmt"># headers</span>
<span class="pykw">print</span> response2.read() <span class="pycmt"># body</span>
response2.close()</pre>
<a name="notes"></a>
<p>All of the standard control types are supported: <code>TEXT</code>,
<code>PASSWORD</code>, <code>HIDDEN</code>, <code>TEXTAREA</code>,
<code>ISINDEX</code>, <code>RESET</code>, <code>BUTTON</code> (<code>INPUT
TYPE=BUTTON</code> and the various <code>BUTTON</code> types),
<code>SUBMIT</code>, <code>IMAGE</code>, <code>RADIO</code>,
<code>CHECKBOX</code>, <code>SELECT</code>/<code>OPTION</code> and
<code>FILE</code> (for file upload). Both standard form encodings
(<code>application/x-www-form-urlencoded</code> and
<code>multipart/form-data</code>) are supported.
<p>The module is designed for testing and automation of web
interfaces, not for implementing interactive user agents.
<p><strong><em>Security note</em>: Remember that any passwords you store in
<code>HTMLForm</code> instances will be saved to disk in the clear if you
pickle them (directly or indirectly). The simplest solution to this is to
avoid pickling <code>HTMLForm</code> objects. You could also pickle before
filling in any password, or just set the password to <code>""</code> before
pickling.</strong>
<p>Python 2.0 or above is required. To run the tests, you need the
<code>unittest</code> module (from <a
href="http://pyunit.sourceforge.net/">PyUnit</a>). <code>unittest</code> is a
standard library module with Python 2.1 and above.
<p>For full documentation, see the docstrings in ClientForm.py.
<p><em><strong>Note: this page describes the 0.2 (stable release)
interface. See <a href="./src/README-0_1_17.html">here</a> for the
old 0.1 interface.</strong> </em>
<a name="parsers"></a>
<h2>Parsers</h2>
<p>ClientForm contains two parsers. See <a href="./#faq">the FAQ entry on
XHTML</a> for details.
<p><a href="http://www.egenix.com/files/python/mxTidy.html">mxTidy</a> or <a
href="http://utidylib.berlios.de/">Tidylib</a> can be useful for dealing with
bad HTML.
<p>I think it would be nice to have an implementation of ClientForm based on <a
href="http://www.crummy.com/software/BeautifulSoup/">BeautifulSoup</a>
(i.e. all methods and attributes implemented using the BeautifulSoup API),
since that module does tolerant HTML parsing with a nice API for doing
non-forms stuff. (I'm not about to do this, though. For anybody interested in
doing this, note that the ClientForm tests would need making
constructor-independent first.)
<a name="compat"></a>
<h2>Backwards-compatibility mode</h2>
<p>ClientForm 0.2 includes three minor backwards-incompatible interface
changes from version 0.1.
<p>To make upgrading from 0.1 easier, and to allow me to stop supporting
version 0.1 sooner, version 0.2 contains support for operating in a
backwards-compatible mode, under which code written for 0.1 should work without
modification. This is done on a per-<code>HTMLForm</code> basis via the
<code>.backwards_compat</code> attribute, but for convenience the
ParseResponse() and ParseFile() factory functions accept
<code>backwards_compat</code> arguments. These backwards-compatibility
features will be removed in version 0.3. The default is to operate in
backwards-compatible mode. To run with backwards compatible mode turned
<em><strong>OFF</strong></em> (<strong>strongly recommended</strong>):
<pre>
<span class="pykw">from</span> urllib2 <span class="pykw">import</span> urlopen
<span class="pykw">from</span> ClientForm <span class="pykw">import</span> ParseResponse
forms = ParseResponse(urlopen(<span class="pystr">"http://example.com/"</span>), backwards_compat=False)
<span class="pycmt"># ...</span></pre>
<p>The backwards-incompatible changes are:
<ul>
<li><p>Ambiguous specification of controls or items now results in
AmbiguityError. If you want the old behaviour, explicitly pass
<code>nr=0</code> to indicate you want the first matching control or item.
<li><p>Item label matching is now done by substring, not by strict
string-equality (but note leading and trailing space is always stripped).
(Control label matching is always done by substring.)
<li><p>Handling of disabled list items has changed. First, note that handling
of disabled list items in 0.1 (and in 0.2's backwards-compatibility mode!) is
buggy: disabled items are successful (ie. disabled item names are sent back to
the server). As a result, there was no distinction to be made between
successful items and selected items. In 0.2, the bug is fixed, so this is no
longer the case, and it is important to note that list controls'
<code>.value</code> attribute contains only the <em>successful</em> item names;
items that are <em>selected </em> but not successful (because disabled) are not
included in <code>.value</code>. Second, disabled list items may no longer be
deselected: AttributeError is raised in 0.2, whereas deselection was allowed in
0.1. The bug in 0.1 and in 0.2's backwards-compatibility mode will not be
fixed, to preserve compatibility and to encourage people to upgrade to the new
0.2 <code>backwards_compat=False</code> behaviour. </ul>
<a name="credits"></a>
<h2>Credits</h2>
<p>Apart from Gisle Aas for allowing the original port from
libwww-perl, particular credit is due to Gary Poster and Benji York,
and their employer, Zope Corporation, for their contributions which
led to ClientForm 0.2 being released. Thanks also to the many people
who have contributed bug reports.
<a name="download"></a>
<h2>Download</h2>
<p>For installation instructions, see the INSTALL.txt file included in the
distribution.
<p><span class="spanhdr">Stable release</span> There have been three fairly
minor backwards-incompatible interface changes since version 0.1 (see <a
href="./#compat">above</a>), but by default the code operates in a
backwards-compatible mode so that code written for 0.1 should work without
changes.
<p>0.2 includes better support for labels, and a simpler interface (all the old
methods are still there, but some have been deprecated and a few added).
<ul>
<li><a href="./src/ClientForm-0.2.10.tar.gz">ClientForm-0.2.10.tar.gz</a>
<li><a href="./src/ClientForm-0.2.10.zip">ClientForm-0.2.10.zip</a>
<li><a href="./src/ChangeLog.txt">Change Log</a> (included in distribution)
<li><a href="./src/">Older releases.</a>
</ul>
<br>
<p><span class="spanhdr">Old release</span> No longer maintained. I recommend
upgrading from 0.1 to 0.2.
<p>There were many interface changes between 0.0 and 0.1, so you should take
care if upgrading old code from 0.0.
<p>0.1 includes <code>FILE</code> control support for file upload, handling
of disabled list items, and a redesigned interface.
<ul>
<li><a href="./src/ClientForm-0.1.17.tar.gz">ClientForm-0.1.17.tar.gz</a>
<li><a href="./src/ClientForm-0_1_17.zip">ClientForm-0_1_17.zip</a>
<li><a href="./src/ChangeLog.txt">Change Log</a> (included in distribution)
<li><a href="./src/">Older releases.</a>
</ul>
<br>
<p><span class="spanhdr">Ancient release</span> No longer maintained. You
don't want this.
<ul>
<li><a href="./src/ClientForm-0.0.16.tar.gz">ClientForm-0.0.16.tar.gz</a>
<li><a href="./src/ClientForm-0_0_16.zip">ClientForm-0_0_16.zip</a>
<li><a href="./src/ChangeLog.txt">Change Log</a> (included in distribution)
<li><a href="./src/">Older releases.</a>
</ul>
<a name="svn"></a>
<h2>Subversion</h2>
<p>The <a href="http://subversion.tigris.org/">Subversion (SVN)</a> trunk is <a href="http://codespeak.net/svn/wwwsearch/ClientForm/trunk#egg=ClientForm-dev">http://codespeak.net/svn/wwwsearch/ClientForm/trunk</a>, so to check out the source:
<pre>
svn co http://codespeak.net/svn/wwwsearch/ClientForm/trunk ClientForm
</pre>
<a name="faq"></a>
<h2>FAQs</h2>
<ul>
<li>Doesn't the standard Python library module, <code>cgi</code>, do this?
<p>No: the <code>cgi</code> module does the server end of the job. It
doesn't know how to parse or fill in a form or how to send it back to the
server.
<li>Which version of Python do I need?
<p>2.0 or above (ClientForm 0.2; version 0.1 requires Python 1.5.2 or above).
<li>Is <code>urllib2</code> required?
<p>No.
<li>How do I use it without <code>urllib2</code>?
<p>Use <code>.click_request_data()</code> instead of <code>.click()</code>.
<li>Which <code>urllib2</code> do I need?
<p>You don't. It's convenient, though. If you have Python 2.0, you need to
upgrade to the version from Python 2.1 (available from <a
href="http://www.python.org/">www.python.org</a>). Otherwise, you're OK.
<li>Which license?
<p>ClientForm is dual-licensed: you may pick either the
<a href="http://www.opensource.org/licenses/bsd-license.php">BSD license</a>,
or the <a href="http://www.zope.org/Resources/ZPL">ZPL 2.1</a> (both are
included in the distribution).
<a name="xhtml"></a>
<li>Is XHTML supported?
<p>Yes. You must pass
<code>form_parser_class=ClientForm.XHTMLCompatibleFormParser</code> to
<code>ParseResponse()</code> / <code>ParseFile()</code>. Note this parser
is less tolerant of bad HTML than the default,
<code>ClientForm.FormParser</code>
<li>How do I figure out what control names and values to use?
<p><code>print form</code> is usually all you need.
In your code, things like the <code>HTMLForm.items</code> attribute of
<code>HTMLForm</code> instances can be useful to inspect forms at
runtime. Note that it's possible to use item labels instead of item
names, which can be useful — use the <code>by_label</code>
arguments to the various methods, and the <code>.get_value_by_label()</code> /
<code>.set_value_by_label()</code> methods on <code>ListControl</code>.
<li>What do those <code>'*'</code> characters mean in the string
representations of list controls?
<p>A <code>*</code> next to an item means that item is selected.
<li>What do those parentheses (round brackets) mean in the string
representations of list controls?
<p>Parentheses <code>(foo)</code> around an item mean that item is disabled.
<li>Why doesn't <some control> turn up in the data returned by
<code>.click*()</code> when that control has non-<code>None</code> value?
<p>Either the control is disabled, or it is not successful for some other
reason. 'Successful' (see HTML 4 specification) means that the control
will cause data to get sent to the server.
<li>Why does ClientForm not follow the HTML 4.0 / RFC 1866 standards for
<code>RADIO</code> and multiple-selection <code>SELECT</code> controls?
<p>Because by default, it follows browser behaviour when setting the
initially-selected items in list controls that have no items explicitly
selected in the HTML. Use the <code>select_default</code> argument to
<code>ParseResponse</code> if you want to follow the RFC 1866 rules
instead. Note that browser behaviour violates the HTML 4.01 specification
in the case of <code>RADIO</code> controls.
<li>Why does <code>.click()</code>ing on a button not work for me?
<ul>
<li>Clicking on a <code>RESET</code> button doesn't do anything, by design
- this is a library for web automation, not an interactive browser.
Even in an interactive browser, clicking on <code>RESET</code> sends
nothing to the server, so there is little point in having
<code>.click()</code> do anything special here.
<li>Clicking on a <code>BUTTON TYPE=BUTTON</code> doesn't do anything
either, also by design. This time, the reason is that that
<code>BUTTON</code> is only in the HTML standard so that one can attach
callbacks to its events. The callbacks are functions in
<code>SCRIPT</code> elements (such as Javascript) embedded in the HTML,
and their execution may result in information getting sent back to the
server. ClientForm, however, knows nothing about these callbacks, so
it can't do anything useful with a click on a <code>BUTTON</code> whose
type is <code>BUTTON</code>.
<li>Generally, embedded script may be messing things up in all kinds of
ways. See the answer to the next question.
</ul>
<li>Embedded script is messing up my form filling. What do I do?
<p>See the <a href="../bits/GeneralFAQ.html">General FAQs</a> page and the
next FAQ entry for what to do about this.
<!-- XXX example here -->
<li>How do I change <code>INPUT TYPE=HIDDEN</code> field values (for example,
to emulate the effect of JavaScript code)?
<p>As with any control, set the control's <code>readonly</code> attribute
false.
<p><pre>
form.find_control(<span class="pystr">"foo"</span>).readonly = False <span class="pycmt"># allow changing .value of control foo</span>
form.set_all_readonly(False) <span class="pycmt"># allow changing the .value of all controls</span></pre>
</li>
<li>I'm having trouble debugging my code.
<p>The <a href="../ClientCookie/">ClientCookie</a> package makes it
easy to get <code>.seek()</code>able response objects, which is
convenient for debugging. See also <a
href="../ClientCookie/doc.html#debugging">here</a> for few
relevant tips. Also see <a href="../bits/GeneralFAQ.html"> General
FAQs</a>.
<li>I have a control containing a list of integers. How do I select the one
whose value is nearest to the one I want?
<p><pre>
<span class="pykw">import</span> bisect
<span class="pykw">def</span> closest_int_value(form, ctrl_name, value):
values = map(int, [item.name <span class="pykw">for</span> item <span class="pykw">in</span> form.find_control(ctrl_name).items])
<span class="pykw">return</span> str(values[bisect.bisect(values, value) - 1])
form[<span class="pystr">"distance"</span>] = [closest_int_value(form, <span class="pystr">"distance"</span>, 23)]</pre>
</li>
<li>Where can I find out more about the HTML and HTTP standards?
<ul>
<li>W3C <a href="http://www.w3.org/TR/html401/">HTML 4.01
Specification</a>.
<li><a href="http://www.ietf.org/rfc/rfc1866.txt">RFC 1866</a> -
the HTML 2.0 standard.
<li><a href="http://www.ietf.org/rfc/rfc1867.txt">RFC 1867</a> -
Form-based file upload.
<li><a href="http://www.ietf.org/rfc/rfc2616.txt">RFC 2616</a> -
HTTP 1.1 Specification.
</ul>
</ul>
<p>I prefer questions and comments to be sent to the <a
href="http://lists.sourceforge.net/lists/listinfo/wwwsearch-general">
mailing list</a> rather than direct to me.
<p><a href="mailto:jjl@pobox.com">John J. Lee</a>,
July 2008.
</div>
<div id="Menu">
<a href="..">Home</a><br>
<br>
<a href="../bits/GeneralFAQ.html">General FAQs</a><br>
<br>
<a href="../mechanize/">mechanize</a><br>
<a href="../mechanize/doc.html"><span class="subpage">mechanize docs</span></a><br>
<span class="thispage">ClientForm</span><br>
<br>
<a href="../ClientCookie/">ClientCookie</a><br>
<a href="../ClientCookie/doc.html"><span class="subpage">ClientCookie docs</span></a><br>
<a href="../pullparser/">pullparser</a><br>
<a href="../DOMForm/">DOMForm</a><br>
<a href="../python-spidermonkey/">python-spidermonkey</a><br>
<a href="../ClientTable/">ClientTable</a><br>
<a href="../bits/urllib2_152.py">1.5.2 urllib2.py</a><br>
<a href="../bits/urllib_152.py">1.5.2 urllib.py</a><br>
<br>
<a href="../#other">Other stuff</a><br>
<br>
<a href="./#example">Example</a><br>
<a href="./#notes">Notes</a><br>
<a href="./#parsers">Parsers</a><br>
<a href="./#compat">Compatibility</a><br>
<a href="./#credits">Credits</a><br>
<a href="./#download">Download</a><br>
<a href="./#faq">FAQs</a><br>
</div>
</body>
</html>
|
