1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
|
README file for KLibido - the KDE Linux Binaries Downloader
Version 0.2.5 - 27/Feb/2005
* What it is:
KLibido is a KDE program to download encoded articles from the usenet news
service, using the nntp protocol. It supports multiple servers, multiple
download threads per server, automatic joining and decoding of articles.
* What it's not:
KLibido is not a NewsReader. It doesn't let you easily display the articles -
only their subject, and it discards all non-binary posts. If you want a nice
newsreader for KDE, try KNode.
* What works:
- Getting the group list and subscribe to newsgroups
- Getting headers, sorting, filtering by subject, state and "completeness"
- downloading and decoding of binary posts (including yEnc).
- Queue management: it's possible to drag and drop items around in the download
queue to change the order of download, pause the queue, pause and cancel the
items in the queue.
- Server priorities and balancing
- Requeueing: if an article fails on a server, it is automatically tried on
the other servers.
- Viewing nfo's, covers, et al inside the program using kparts.
- Group categories and aliases
- .nzb support
* What doesn't work:
- No documentation :)
- There are probably a lot of bugs. Having said that, it quite works, and I've
used it to download tens of gigabytes.
* How does it do it:
- KLibido uses kde and the kde libraries for the interface, Berkeley DB for
storing the newsgroups' headers, and UUDeview for decoding the posts.
Requirements:
- Kde/Qt 3.2 or greater
- Berkeley Db 4.x (http://www.sleepycat.com/products/db.shtml) with C++
support. If you compile it from source, remember to pass the "--enable-cxx"
option to configure.
- UUdeview library version 0.5.20 (http://www.fpx.de/fp/Software/UUDeview/).
If you're a Gentoo user, emerge dev-libs/uulib.
* How to compile:
- You can grab the source from sourceforge (go to
http://www.sf.net/projects/klibido). It compiles as usual:
./configure --prefix=<your kde prefix>
make
make install
You can find out your kde prefix running
kde-config --prefix
PLEASE NOTE: Lots of problems/crashes have been reported when using
optimizations. Please be conservative.
If you want to grab the source from CVS, go to
http://sourceforge.net/cvs/?group_id=114928 and follow the instructions.
PLEASE NOTE: I try to commit only working code, but the CVS can be very
unstable at times. Always read the Changelog before compiling/using.
* Binary Packages:
On the program's homepage there are links/instructions for downloading binary packages for some distributions (Mandrake, Fedora, SuSE, Debian, etc...). See http://klibido.sf.net/ if you don't want to compile the source yourself.
* Getting started:
Type "klibido" to start the program. Sorry, no fancy wizards at the moment. To
get it working:
- Configure the directories (a default is provided)
- Add your servers (Server->new server)
- Get the group list (Server->get list of groups)
- Subscribe to some groups (newsgroup->subscribe to newsgroup)
- NewsGroup->update subscribed groups
- Enjoy :)
* A note about error model:
KLibido divides errors in two branches, fatal (i.e.: article not found) and not
fatal (i.e.: timeout). Fatal errors cause the download of the item to fail,
while non-fatal errors trigger a retry.
In the server options, there are three values that control the connection
timeout/errors:
1. Server timeout: this is the value of timeout for the server (in seconds),
i.e.: if no response/data come from the server in "timeout" seconds, a timeout
error is triggered (and the item is generally retried)
2. Retries: this value only applies to post downloads (i.e.: not to headers
updates) and represents the number of times a download is retried in the event
of a *fatal* error. Example: KLibido tries to download an article. The server
answers "article not found". The download is retried "Retries" times
(default: 3). Why, you may ask? Because some servers are lame/overloaded/whatever,
and when you try to download an article you get "article not found" the first time,
but if you retry in a few seconds, the download succeeds. If your servers don't
show this behavior, put "retries" to 0.
3. Thread timeout: this is the timeout (in minutes) for the connection thread.
When a thread fails to successfully connect/login to a server for "thread_timeout"
minutes, the thread is paused and must be restarted manually (via the context menu).
This is considered a communication error, so it doesn't cause any item in the queue
to fail.
More in depth: when a thread gets a login error, it is paused for n seconds.
"n" depends on the number of *consecutive* fails: first time 1 seconds, second
5 seconds, third 10 seconds, then 30, then one minute. After that, the thread
is always paused for one minute. After "thread_timeout" minutes, the thread is
put in paused state, and doesn't try to connect anymore until the user restarts
it manually. Why? Because a login error can have to much causes:
- network is down
- wrong login/password
- thread limit exceeded
- download limit exceeded
- etc...
Servers often return custom error codes, so in the event of a failed login there
is no easy/automated way to know if the login failed because of a wrong password,
exceeded daily download limit, too many connected threads, or whatever. So the best
thing to do (IMHO) is to pause the thread and retry later.
* The story behind it:
The main goals I had in mind while writing it were:
- Keeping memory usage as low as possible. Update/load/browse large groups
(groups with over one million of articles) without using 1 Gb of RAM and
swapping for days.
- Multiple server support
- Multiple download threads support
- Fine-grained manipulation of the download queue.
I hope to have succeeded.
* Who am I:
- Does anyone care? :-]
* Credits
- KNode and Pan developers for letting me steal their icons :)
- Sleepycat for the great Berkeley Db
- Frank Pilhofer for the UUDeview Library.
- The KDevelop team for the great IDE they've made.
- The KDE developers for the great GUI.
- David Pye (.nzb parsing, yy/uu decoder)
* Related software you will find on the 'net:
- nget - http://nget.sourceforge.net/ - The *best* command-line nntp grabber.
- BNR - http://www.bnr2.org/ - Multi-platform nntp grabber, available for Linux
and Windows.
- Pan - http://pan.rebelbase.com/ - Great newsreader with binary skills :).
- Knzb - http://dmp.org.uk/linux/knzb/ - KDE .nzb (Nezbin) downloader
(C) 2004/2005 Alessandro "Bauno" Bonometti, <bauno_at_inwind.it>
|
