1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
|
TODO for IMDbPY
===============
See the code, and search for XXX, FIXME and TODO.
NOTE: it's always time to clean the code! <g>
[general]
* Write better summary() methods for Movie and Person classes.
* Some portions of code are poorly commented.
* The documentation is written in my funny English.
* Compatibility with Python 2.2 and previous versions is no more assured
for every data access system (the imdbpy2sql.py script for sure
requires at least Python 2.3).
* The analyze_title/build_title functions are grown too complex and
beyond their initial goals.
* the 'year' keyword can probably be an int, instead of a string;
the '????' case can be handled directly by the analyze_title/build_title
functions.
* for local and sql data access systems: some episode titles are
marked as {{SUSPENDED}}; they should probably be ignored.
[searches]
* Support advanced query for movie titles/person names.
[Movie objects]
* Define invariable names for the sections (the keys you use to access
info stored in a Movie object).
* Should a movie object automatically build a Movie object, when
a 'episode of' dictionary is in the data?
* For TV series the list of directors/writers returned by 'local'
and 'sql' is a long list with every single episodes listed in the
'notes' attribute (i.e.: the same person is listed more than one time,
just with a different note).
For 'http' and 'mobile' there's a list with one item for every
person, with a long 'notes' listing every episode.
It's not easy to split these information since they can contain
notes ("written by", "as Aka Name", ...)
* The 'laserdisc' information for 'local' and 'sql' is probabily
wrong: I think they merge data from different laserdisc titles.
* there are links to hollywoodreporter.com that are not bathered in
the "external reviews" page.
[Person objects]
* Define invariable names for the sections (the keys you use to access
info stored in a Person object).
* Fetching data from the web ('http' and 'mobile'), the filmography
for a given person contains a list named "himself" or "herself" for
movies/shows where they were not acting.
In 'local' and 'sql', these movies/shows are listed in the
"actor" or "actress" list.
Check if this is still true, with the new IMDb's schema.
[http data access system]
* There's a known bug using IMDbPY with Python 2.0 and 2.1:
it fails to retrieve web pages, raising the httplib.UnknownTransferEncoding
exception, if a proxy is used; unset the HTTP_PROXY environment variable
or call the imdbObject.set_proxy(None) method.
* If the access through the proxy fails, is it possible to
automatically try without? It doesn't seem easy...
* Some (many?) HTML parser can be interrupted as long as they've
parsed every needed information, as HTMLSearchMovieParser does.
* Access to the "my IMDb" functions for registered users would
be really cool.
* Gather more movies' data: user comments, laserdisc details, trailers,
posters, photo gallery, on tv, schedule links, showtimes, message boards.
* Gather more people's data: photo gallery.
[httpThin data access system]
* It should be made _really_ faster than 'http'.
[mobile data access system]
* General optimization.
* Make find() methods case insensitive.
[local data access system]
* There's probably a bug converting the rating to a float;
see: ER (1994) (TV), but I suspect this to be a mkdb bug.
* The 'votes' key is not correctly stored for very high values;
this is for sure a mkdb bug.
* Files like biographies.data are plainly wrong: sometimes
a person is referred with 'Name Surname' (qv) and sometimes
with 'Surname, Name' (qv).
Names and titles with "'" are not handled properly and so on...
This is a problem of the Plain Text Data Files, and can't be fixed.
* The convBin() and getFullIndex() functions in the utils
modules _must_ be rewritten (they're crap!).
* Some files are not considered: mpaa-ratings-reasons.list,
miscellaneous-companies.list (but they're used by the 'sql' data
access system).
* You need the mkdb executable from the moviedb program to generate the
database/index files; moviedb is not open source software (although
it can be downloaded and used without paying money).
An open source (pure python?) script must be written.
NOTE: since release 2.1 I've introduced the imdbpy2sql.py script,
so a replacement for mkdb is no more an urgent issue.
[sql data access system]
NOTE NOTE NOTE: this is still beta code and I'm not a database guru;
moreover I'm short of time and so I will be happy to fix every bug
you'll find, but if you're about to write me an email like "ehi,
the database access should be faster", "the imdbpy2sql.py script must
run with 64 MB of RAM and complete in 2 minutes" or "your database
layout sucks: I've an idea for a better structure...", well, consider
that _these_ kinds of email will be probably immediately discarded.
I _know_ these are important issues, but I've neither the time nor
the ability to fix these problems by myself, sorry.
Obviously if you want to contribute with patches, new code, new
SQL queries and new database structures you're welcome and I will
be very grateful for your work.
Again: if there's something that bother you, write some code.
It's free software, after all.
Things to do:
* The imdbpy2sql.py script MUST be run on a database with empty tables;
unfortunately so far a SQL installation can't be "updated" without
recreating the database from scratch.
IMDb releases also "diff" files to keep the plain text files updated;
it would be wonderful to directly use these diff files to upgrade the
SQL database, but I think this is a nearly impossible task.
|
