1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713
|
F*EX (Frams' Fast File EXchange) is a service to send big (large, huge, giant,
...) files from a user A to a user B, anywhere on the internet.
The sender uploads the file to the F*EX-server and the recipient automatically
gets a notification email with a download-URL.
Main features of F*EX
* file transfer of virtually unlimited file size
* recipient and sender only need an email program and a web browser - of any
kind, they do not have to install any software
* RESEND and REGET for resuming after link failures at last sent byte
* auto-notification of recipient
* auto-deletion after download
* auto-deletion after expiration date (default: 5 days)
* full-users can create one time upload URLs for foreign users
* full-users can create subusers, who can send only to this full-user
* full-users can create groups, an analogy to mailing lists, but for files
* self registration possible for internal and external users
* maintenance-free: no admin interaction necessary
* sending to multiple recipients needs storage on the server only once
* quotas for sending and receiving
* F*EX is a HTTP web-service and needs no firewall-tunnels
* support for (personal or public) streams, too (SEX : Stream EXchange)
* for real UNIX users, there are the shell programs fexsend and fexget to
avoid annoying web browser usage and for full scripting support
* adminstration by CLI or Web
* about 10 times faster than apache
* authentification modules for LDAP, RADIUS, POP, mailman
* server based user address books
* (reverse) proxy support
* "public upload" similar to "anonymous ftp" possible
* "anonymous upload" without registration for enabled IP addresses possible
* localization for many languages
* authentification by local user database, RADIUS, LDAP, mailman or POP
* user and admin can throttle bandwith usage
* admin can restrict access based on email or IP addresses
* server available for UNIX and Windows hosts
* protocol and source-code free available
The end user normally uses F*EX with his web browser and the URLs
http://YOURFEXSERVER/fup (file upload) and http://YOURFEXSERVER/fop (file
download).
F*EX is not an anonymous service (exception: "public upload" and
"anonymous upload", see later).
The admin must register the user with his email address and auth-ID
string (low security password). This task can be done either with the CLI
program "fac" (F*EX admin control) or http://YOURFEXSERVER/fac
Alternativly the users can register theirselves with http://YOURFEXSERVER/fur
(F*EX user registration), if the admin allows them to do so. This is done by
setting the variables @local_domains and @local_hosts in $FEXHOME/lib/fex.ph
Example:
@local_hosts = qw(127.0.0.1 10.10.100.0-10.10.255.255);
@local_domains = qw(rus.uni-stuttgart.de flupp.org);
You can also use wildcard "*" in a domain, for example:
@local_domains = qw(uni-stuttgart.de *.uni-stuttgart.de);
Then all subdomains are allowed.
In addition the administrator can allow external users to fex (only) to
internal users. The F*EX functioning scheme is close related to email
where any external user can send email to an internal user. This is also
possible with F*EX. These users are then called "restricted users". The
administrator has to define via configuration variable @local_rdomains
the local recipient domains. For example:
@local_rdomains = qw(YOURDOMAIN *.YOURDOMAIN OTHERDOMAIN);
F*EX full users can create one time upload URLs with
http://YOURFEXSERVER/fuc
With such a URL a foreign user can send this F*EX full user a single file.
F*EX full users can theirselves register "subusers" with
http://YOURFEXSERVER/fuc
Subusers can only fex to their full-user, not to any others, and they
cannot create other subusers.
F*EX full users also can create groups, an analogy to mailing lists, but
for files: every member of a group can send files to the group address.
The administrator can set with fac the CAPTIVE flag for a full user. Then
he cannot change anymore his default settings (KEEP, AUTODELETE, etc)
The F*EX user is the same as the "sender" in the fup CGI and the "from"
parameter in the F*EX URL.
The (confusing) naming scheme is historically based :-)
The recipient (normally) does not need any registration. He authenticates
himself with his unique download-URL which he gets in the notification
email.
You do not need to build F*EX URLs manually, they are generated by the
F*EX programs.
A F*EX session is based on unique URL parameters or POST variables (FROM,
TO, ID, etc). For security reasons the URL parameters FROM and ID become a
MD5-hashed access key link (AKEY for regular users, SKEY for subusers,
GKEY for groupusers), generated by the CGIs, to prevent that a third
person could easily read these parameters from the user webbrowser URL
address field. AKEY and SKEY are used as optional cookies, too, for
extending the session life.
A SKEY is made of md5_hex("$mainuser:$subuser:$subuserid")
A GKEY is made of md5_hex("$mainuser:$groupname:$groupuser:$groupuserid")
Note: the AKEY, SKEY and GKEY always can be stolen by a network sniffer!
If you need true security, then you have to use https instead of http!
After download the file will be deleted after a grace time of 1 minute.
This grace time allows a recipient to get the file again if he had
problems in saving it.
With the fexsend client the sender can change this behavior:
option -D means "delay autodelete": do not delete the the file directly
after download, but with the nightly fex_cleanup cronjob. More downloads
are possible only from the same client (identified by cookie or ip
address).
option -K means "keep file": do not delete the file after download, but
only after expiration date (normally 5 days). More downloads are possible
only from the same client (identified by cookie or ip address).
If you fex a file to yourself (sender = recipient), then the resulting
download link is valid for any client and can be downloaded everywhere
and more than once (until expiration date).
If you want "delay autodelete" to be the default behaviour for all users
and each transfer then set
$autodelete = 'DELAY'; # or 'NO' for no autodelete
in $FEXHOME/lib/fex.ph
In addition, you can add to the "Recipient(s)" field of the fup CGI:
":autodelete=delay" or ":autodelete=no" or ":keep=x" (where x is the number
of days). Example:
Recipient(s): framstag@rus.uni-stuttgart.de:keep=10
These options are also possible in the server address book (see CGI fuc).
If you need more security, then set in fex.ph:
$fop_auth = 'yes';
$force_https = 'yes';
With $fop_auth upload is restricted to registered users and download
requires (HTTP) authorization. The credentials are the F*EX user email
and auth-ID.
See extra documentation SSL for using https.
If you want to have encrypted emails, then you need a GPG public key for
the user fex. Create it with "gpg --gen-key" (use fex@YOURFEXSERVER as the
user or the $bcc address from fex.ph if you have set it, see below). Next,
the user has to upload his public key via webinterface "user config &
operation control" and "Manage e-mail encryption".
"Public upload" is a feature similar to anonymous ftp: the sender does not
to be registered, but may send only to predefined recipients via
http://YOURFEXSERVER/pup
The administrator has to define via configuration variable
@public_recipients the email addresses for which anyone can upload files
without registration (auth-ID is not necessary). For example:
@public_recipients = qw(fexmaster@YOURDOMAIN bunny@OTHERDOMAIN);
With "anonymous upload" one can use F*EX without registration or
authentication. The administrator has to define via configuration variable
@anonymous_upload for which IP addresses this feature is available.
The administrator can globally forbid certain recipient address via fex.ph
configuration (example):
@forbidden_recipients = qw(nobody@* *@microsoft.com);
The administrator can also forbid a user to fex to any recipient address,
but the allowed ones with: fac -r USER
By standard installation the base directory $FEXHOME is the same as the
login HOME of user fex, but you can move it if you want. $FEXHOME is
determined by the full path of fexsrv as configured in
/etc/xinetd.d/fex . Change this when you move $FEXHOME!
You can also add (name based) virtual hosts with fac.
Do not give write permission to any other user to any file in $FEXHOME or
below!
$FEXHOME contains:
spool/ spool directory and user data
htdocs/ directory for generic download files
bin/ directory for programs
cgi-bin/ directory for CGI programs
lib/ directory for library and config files
doc/ additional documentation
locale/ alternative language installation
Files in spool:
cleanup.log log of daily cleanup cronjob
dkey.log log of download keys
dop.log log of HTTP document output
error.log log of errors
fexsrv.log log of all HTTP requests
fop.log log of file downloads
fup.log log of file uploads
fur.log log of user self registrations
sex.log log of stream exchanges
$user/@ regular user auth-ID
$user/@SUBUSER subuser addresses and IDs
$user/@ALLOWED_RECIPIENTS recipients restrictions for this user
$user/@ALLOWED_RHOSTS recipient's hosts restrictions
$user/@UPLOAD_HOSTS upload hosts restrictions
$user/@DOWNLOAD_HOSTS download hosts restrictions
$user/@ADDRESS_BOOK users recipient address book
$user/@GROUP directory of F*EX user groups
$user/@OKEY directory with one time upload keys
$user/@QUOTA sender and recipient quotas
$user/@AUTODELETE autodelete default
$user/@KEEP keep default
$user/@LOCALE locale default
$user/@CAPTIVE user must not change his settings
$user/@FEXYOURSELF user can only fex to himself via
web interface
$to/$from/$file/upload file data in upload progress
$to/$from/$file/filename original file name
$to/$from/$file/size original file size
$to/$from/$file/useragent HTTP header User-Agent
$to/$from/$file/uurl upload URL
$to/$from/$file/data file data after complete upload
$to/$from/$file/keep keep time (autoexpire) in days
$to/$from/$file/autodelete autodelete option: YES NO or DELAY
$to/$from/$file/comment comment in notification email
$to/$from/$file/id file ID (optional)
$to/$from/$file/ip sender ip address
$to/$from/$file/notify reminder notify flag
$to/$from/$file/error error message if file has gone
$to/$from/$file/download log of successful downloads
$to/$from/$file/restrictions IP based download restrictions
(see $user/@ALLOWED_RHOSTS)
$to/$from/$file/dkey download key
$to/$from/$file/locale locale
As you see, the first directory sometimes means sender and sometimes means
recipient! It depends on the case of usage (up/download).
BEWARE: for the special senders fexmail and anonymous $to and $from are
swapped (by fup and fop)! This is to prevent overcrowding of the toplevel
spool directory with _fexmail_$RANDOM and _anonymous_$RANDOM recipients.
A registered full F*EX user is identified by the file $spooldir/$from/@
Only if this file contains his auth-ID this user is able to send files to
others. Otherwise he is just an unpriviledged recipient.
You can customize the upload CGI fup by editing $FEXHOME/lib/fup.pl
Additional directories in spool:
.dkeys/ download keys lookup directory
.ukeys/ upload keys lookup directory
.akeys/ authentication keys lookup directory
.skeys/ subuser keys lookup directory
.gkeys/ groupuser keys lookup directory
.xkeys/ extra download keys lookup directory
.debug/ directory for debug output (optional)
.reg/ temporary data for user selfregistration
The upload keys (UKEY) are necessary to synchronize the file upload by
HTTP POST with the javascript upload status bar which is an asynchronous
HTTP GET. The GET process needs to know the corresponding POST process.
This is done via UKEY, a unique upload identifier.
The download key (DKEY) is a unique identifier for - guess what -
downloading. It also prevents an attacker to get the file, because only
the recipient knows the DKEY as part of the download URL from the
notification email.
XKEY is an optional extra download key to have a short download URL in
shape http://YOURFEXSERVER//XKEY
The XKEY always starts with // and is provided as a COMMENT by the client.
Example: fexsend schwuppdiwupp.jpg //
The user has to realize that such URLs have very low security.
If you need to trace a F*EX request, then set
$debug = 1;
in fex.ph and look in ~/spool/.debug/ for the correspondening files.
The user fex also gets a Bcc of all sent notification emails. This is a
help for debugging purposes. If you want these Bcc mails to another
address then set in fex.ph:
$bcc = 'other@address';
The sender also get a Bcc if he put the string "!bcc!" in the comment
field.
If you have users within another domain with its own MTA and those users
want to fex each other, then their MTA probably will reject the
notification email. To prevent this problem you can set in fex.ph
@remote_domains = qw(other_domain1 other_domain2 ...);
In this case "From: $admin" will be used in the notification emails.
F*EX comes with its own web server: fexsrv
Standard web servers like apache have been proven problematic, either in
speed or because of a 2-GB-limit.
It is not possible to use the F*EX CGIs with an alternative web server,
because the F*EX CGIs need special fexsrv features.
xinetd starts fexsrv for every new connection, which then executes the
CGIs fup (file upload), fop (file output), fuc (fex user control), foc
(fex operation control), fac (fex admin control), fur (fex user
registration) and sex (stream exchange).
SEX has the opposite authorization model of FEX: The sender does not
need any registration. He can send a stream to any registered SEX user,
who must identify himself with his auth-ID. This is because a stream does
not cost resources when it is not bound to a recipient. It will block
otherwise. The client programs for sexing are sexsend and sexget. You can
call them also fuckyou and fuckme :-)
With the F*EX client fexsend you can have a streaming file transfer with
spooling: on client side there is no temporary buffer file (archive), but
the data is sent directly to the F*EX server. This saves time and disk
space on the client.
Example for a streaming archive upload:
fexsend -a music+video.tar *mp3 *avi webmaster@flupp.org
Example for a streaming data upload:
fpg fex /var/log/* | fexsend -s logs fexmaster@flupp.org
The data streaming is implemented via a HTTP extension: the header
Content-Length is given -1 which means: unknown length, read until EOF.
For streaming receiving you can use "fexget -s-" or "wget -O-".
fexsrv also can do generic document output (via dop) like a normal web
server. For this, your files must be under $FEXHOME/htdocs and they must
not have the same name as the CGIs under $FEXHOME/cgi-bin, because the CGIs
have priority.
For security reasons, documents to be delivered by dop:
- the file must be readable by group or world
- the file must be in $FEXHOME/htdocs or a directory specified by @doc_dirs
- the filename must not start with a "."
- the filename must not contain a "@"
- the filename must not end with "~"
To enable HTTP basic authentication, write your access token to a file
named .htauth which will protect all files in this directory. An user will
be prompted for this password by his web browser.
To restrict the access to specific client IP addresses, put these IPs into
a file named .htaccessfrom which will protect all files below this
directory. You can name single IPs, also as IP ranges (example:
129.69.0.0-129.69.255.255)
dop is not a regular CGI program (like fup or fop), but a sub-program of
fexsrv.
*.html files may contain $VARIABLES$ which will be substituted with the
value of the corresponding environment variable. See example
$SERVER_ADMIN$ in $FEXHOME/htdocs/index.html
*.html files may contain <<perl-code>> (even multiline) which will be
evaluated. The output from print and printf statements will be placed
in. If the perl-code does not end with a ";" then its return value is also
added to the output.
Same goes for <<<perl-code>>> but without output catching.
This perl-code must not contain ">>" strings itself!
See example $FEXHOME/htdocs/dynamic.html
To chainload external perl-code do not use "require" but "do" statement.
Pay attention: do not place security relevant data inside << >> because it
will be delivered to the client if the URL ends with a "!" character, see
example:
http://fex.rus.uni-stuttgart.de/index.html
http://fex.rus.uni-stuttgart.de/index.html!
If you want to hide your code, then you have to write a CGI script and
place it into the cgi-bin directory. Files there will never be delivered
by dop. You also can do "chmod o-r file.html" to prevent source code
delivery.
*.html files may contain conditional blocks:
#if (perl-expression)
HTML...
#elseif (perl-expression)
HTML...
#else
HTML...
#endif
which will be evaluated at run-time (#else and #elseif are optional).
*.html files may contain:
#include "FILE"
which means: load FILE from current directory at this place in code.
dop also can display a directory listing if this directory contains a file
named .htindex
To restrict the listing to specific files, put a matching regular
expression into .htindex
Example: echo 'pdf$' > /home/fex/htdocs/doc/.htindex
dop can output gzipped files on-the-fly as a streaming document. For
example: you have /fex/home/htdocs/big.file but a client requests
http://YOURFEXSERVER/big.file.gz then a corresponding big.file.gz will be
delivered.
dop can also output tar, tgz or zip archives on-the-fly as a streaming
document. Just create a symbolic link with files (or directories) you want
in this archive and enclose them with ":". The symlink name must end with
".stream". Example:
http://fex.rus.uni-stuttgart.de:8080/download/sex.tar
fex@fex:~/htdocs/download: ls -l sex*
lrwxrwxrwx 1 fex fex 7 2011-01-26 11:46 sexget -> sexsend
-rwxr-xr-x 1 fex fex 12110 2012-08-22 09:55 sexsend
lrwxrwxrwx 1 fex fex 22 2012-09-01 12:13 sex.stream -> :sexsend:sexget:sexxx:
lrwxrwxrwx 1 fex fex 7 2012-09-01 11:28 sexxx -> sexsend
It is also possible to create this stream file as a regular file. Then the
content must be the file names you want in the streaming archive.
Note: you may only use relative paths and without "../" elements.
Note: Files beginning with a . or ending with ~ will not be included in
the download stream.
cronjob fex_cleanup is run once a day and deletes expired uploads, removes
inactive accounts and does some other spool houskeeping. See: crontab -l
To detect inactive users you can put in fex.ph:
$account_expire = "365:notify";
This sends an email to $admin when an user is inactive for 365 days.
Or to delete inactive users you can put in fex.ph:
$account_expire = "365:delete";
This deletes user accounts automatically which have been inactive for 365
days.
Of course you can use any number of days.
Inactive users will get an account reactivation request before.
The address book may contain the optional parameters autodelete and
keep. Address book example entry:
flupp framstag@flupp.org keep=99 autodelete=no
BUT: these address book parameters are only accepted when using the
fexsend or schwuppdiwupp client! They will be ignored when using a
webbrowser!
Nevertheless you may add autodelete and keep parameters using a
webbrowser, but then they are valid for ALL addresses in the "Recipient"
field!
The clients fexsend and schwuppdiwupp can MD5-encrypt the auth-ID together
with a session-ID. The session-ID itself is provided by the server after
an initial "GET SID" HTTP request. This is not possible through a proxy,
because most proxies do not support persistant tcp sessions.
See fex-client_2.pdf for the F*EX protocol specification.
To understand and trace the F*EX protocol you can use fexsend with the -v
option.
Example (--> means send to server, <-- means receive from server):
framstag@fex:~: fexsend -v X.png framstag@flupp.org
ID data from /home/framstag/.fex/id: http://fex.rus.uni-stuttgart.de framstag@rus.uni-stuttgart.de XXXX
Server/User: http://fex.rus.uni-stuttgart.de/framstag@rus.uni-stuttgart.de
TCPCONNECT to fex.rus.uni-stuttgart.de
--> GET SID HTTP/1.1
-->
<-- HTTP/1.1 201 8p2Y2qa2
<-- X-Features: SID,CHECKRECIPIENT,GROUPS,QUOTA,FILEID,MULTIPOST,XKEY
<-- X-SID: 8p2Y2qa2
<-- X-Timeout: 30
<-- Content-Length: 0
<--
--> HEAD /fop/framstag@flupp.org/framstag@fex.rus.uni-stuttgart.de/X.png??ID=MD5H:226e896d0adab86892957aa8158b37ba HTTP/1.1
-->
<-- HTTP/1.1 200 OK
<-- Content-Length: 0
<-- X-Size: 0
<-- X-Features: SID,CHECKRECIPIENT,GROUPS,QUOTA,FILEID,MULTIPOST,XKEY
<--
--> POST /fup HTTP/1.1
--> Host: fex.rus.uni-stuttgart.de
--> User-Agent: fexsend (Ubuntu 8.04.4 LTS)
--> Content-Length: 149935162
--> Content-Type: multipart/form-data; boundary=JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs
-->
--> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs
--> Content-Disposition: form-data; name="FROM"
-->
--> framstag@fex.rus.uni-stuttgart.de
--> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs
--> Content-Disposition: form-data; name="TO"
-->
--> framstag@flupp.org
--> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs
--> Content-Disposition: form-data; name="ID"
-->
--> MD5H:226e896d0adab86892957aa8158b37ba
--> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs
--> Content-Disposition: form-data; name="FILESIZE"
-->
--> 149934400
--> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs
--> Content-Disposition: form-data; name="FILE"; filename="X.png"
--> Content-Type: application/octet-stream
--> Content-Length: 149934400
--> X-File-ID: 1283077463
-->
--> (file content)
--> --JhUOtQ3sgV5ZcHJzrTny523nBFqgUNvSAOUHoRMTdZfGpAjs--
<-- Location: http://fex.rus.uni-stuttgart.de/fop/CoVhikzk/X.png
<-- X-Recipient: framstag@flupp.org (autodelete=YES,keep=5)
<--
Comment on the HEAD request above:
The client fexsend sends it to request whether the file has been sent
before and if it was successful or not. On the later case, the server
would reply how many bytes has already been received and the client then
can send only the missing part. Normally the answer to the HEAD request is
0 (see above), which means: nothing of this file has been received so far.
But this is only half of the truth :-)
I have omitted in the example above the requests CHECKRECIPIENT and
GET ADDRESS_BOOK, to keep it simple in the first run.
CHECKRECIPIENT is a HTTP POST request to check whether the recpient's
email address is valid to the server.
GET ADDRESS_BOOK is a HTTP GET request to check if the recipient with a
short address (= address without @) is an alias in the senders server
address book.
Please use "fexsend -v" by yourself to see the whole protocol dialoge.
Many HTTP proxies have a POST limit, which is often at 2 GB, but sometimes
lower. To overcome this limitation a F*EX client (like fexsend) may
include in the POST the HTTP MIME part "filesize" which indicates the full
file size. All incoming file POSTs with this file name will be appended
until the full file size is reached. Of course the client has to POST all
parts of this file, in the correct order, in asking the server how many
bytes has already been received. Example:
framstag@fex:/tmp: fexsend -vP wwwproxy.uni-stuttgart.de.de:8080:1024 2GB.tmp .
ID data from /home/framstag/.fex/id: http://fex.rus.uni-stuttgart.de:8080 framstag@rus.uni-stuttgart.de xxxxxx
Server/User: http://fex.rus.uni-stuttgart.de:8080/framstag@rus.uni-stuttgart.de
TCPCONNECT to wwwproxy.uni-stuttgart.de.de:8080
--> HEAD http://fex.rus.uni-stuttgart.de:8080/fop/framstag@rus.uni-stuttgart.de/framstag@rus.uni-stuttgart.de/2GB.tmp??&ID=xxxxxx HTTP/1.1
-->
<-- HTTP/1.1 200 OK
<-- X-Size: 0
<-- Content-Length: 0
<-- Proxy-Connection: Keep-Alive
<-- Connection: Keep-Alive
<-- Date: Tue, 08 Jun 2010 15:04:13 GMT
<--
TCPCONNECT to wwwproxy.uni-stuttgart.de.de:8080
--> POST http://fex.rus.uni-stuttgart.de:8080/fup HTTP/1.1
--> Host: fex.rus.uni-stuttgart.de:8080
--> User-Agent: fexsend (Ubuntu 8.04.4 LTS)
--> Content-Length: 1073677143
--> Content-Type: multipart/form-data; boundary=gqhqeU7FCHMl4ohxJ5knK1y9rF75xYTvabPvZTeWnAhaoDsK
-->
--> --gqhqeU7FCHMl4ohxJ5knK1y9rF75xYTvabPvZTeWnAhaoDsK
--> Content-Disposition: form-data; name="FROM"
-->
--> framstag@rus.uni-stuttgart.de
--> --gqhqeU7FCHMl4ohxJ5knK1y9rF75xYTvabPvZTeWnAhaoDsK
--> Content-Disposition: form-data; name="TO"
-->
--> framstag@rus.uni-stuttgart.de
--> --gqhqeU7FCHMl4ohxJ5knK1y9rF75xYTvabPvZTeWnAhaoDsK
--> Content-Disposition: form-data; name="ID"
-->
--> xxxxxx
--> --gqhqeU7FCHMl4ohxJ5knK1y9rF75xYTvabPvZTeWnAhaoDsK
--> Content-Disposition: form-data; name="COMMENT"
-->
--> NOMAIL
--> --gqhqeU7FCHMl4ohxJ5knK1y9rF75xYTvabPvZTeWnAhaoDsK
--> Content-Disposition: form-data; name="FILESIZE"
-->
--> 2147483748
--> --gqhqeU7FCHMl4ohxJ5knK1y9rF75xYTvabPvZTeWnAhaoDsK
--> Content-Disposition: form-data; name="FILE"; filename="2GB.tmp"
--> Content-Type: application/octet-stream
--> Content-Length: 1073676288
--> X-File-ID: 1275549886
-->
--> (file content)
--> --gqhqeU7FCHMl4ohxJ5knK1y9rF75xYTvabPvZTeWnAhaoDsK--
2GB.tmp : 1023 MB in 73 s (14363 kB/s), chunk #1 : 1023 MB
<-- HTTP/1.1 206 Partial OK
<-- X-Message: Partial OK
<-- Server: fexsrv
<-- Expires: 0
<-- Cache-Control: no-cache
<-- Content-Type: text/html; charset=UTF-8
<-- Connection: close
<-- Date: Tue, 08 Jun 2010 15:05:29 GMT
<--
TCPCONNECT to wwwproxy.uni-stuttgart.de.de:8080
--> HEAD http://fex.rus.uni-stuttgart.de:8080/fop/framstag@rus.uni-stuttgart.de/framstag@rus.uni-stuttgart.de/2GB.tmp??&ID=xxxxxx HTTP/1.1
-->
<-- HTTP/1.1 200 OK
<-- X-Size: 2147483748
<-- X-File-ID: 1275549886
<-- Content-Length: 1073676288
<-- Proxy-Connection: Keep-Alive
<-- Connection: Keep-Alive
<-- Date: Tue, 08 Jun 2010 15:05:29 GMT
<--
TCPCONNECT to wwwproxy.uni-stuttgart.de.de:8080
--> POST http://fex.rus.uni-stuttgart.de:8080/fup HTTP/1.1
--> Host: fex.rus.uni-stuttgart.de:8080
--> User-Agent: fexsend (Ubuntu 8.04.4 LTS)
--> Content-Length: 1073677254
--> Content-Type: multipart/form-data; boundary=LofreCkx1GmgWNCHZ9exBQoQhiEF7pXSaPp2KUQVPlnarPAw
-->
--> --LofreCkx1GmgWNCHZ9exBQoQhiEF7pXSaPp2KUQVPlnarPAw
--> Content-Disposition: form-data; name="FROM"
-->
--> framstag@rus.uni-stuttgart.de
--> --LofreCkx1GmgWNCHZ9exBQoQhiEF7pXSaPp2KUQVPlnarPAw
--> Content-Disposition: form-data; name="TO"
-->
--> framstag@rus.uni-stuttgart.de
--> --LofreCkx1GmgWNCHZ9exBQoQhiEF7pXSaPp2KUQVPlnarPAw
--> Content-Disposition: form-data; name="ID"
-->
--> xxxxxx
--> --LofreCkx1GmgWNCHZ9exBQoQhiEF7pXSaPp2KUQVPlnarPAw
--> Content-Disposition: form-data; name="COMMENT"
-->
--> NOMAIL
--> --LofreCkx1GmgWNCHZ9exBQoQhiEF7pXSaPp2KUQVPlnarPAw
--> Content-Disposition: form-data; name="SEEK"
-->
--> 1073676288
--> --LofreCkx1GmgWNCHZ9exBQoQhiEF7pXSaPp2KUQVPlnarPAw
--> Content-Disposition: form-data; name="FILESIZE"
-->
--> 2147483748
--> --LofreCkx1GmgWNCHZ9exBQoQhiEF7pXSaPp2KUQVPlnarPAw
--> Content-Disposition: form-data; name="FILE"; filename="2GB.tmp"
--> Content-Type: application/octet-stream
--> Content-Length: 1073676288
--> X-File-ID: 1275549886
-->
--> (file content)
--> --LofreCkx1GmgWNCHZ9exBQoQhiEF7pXSaPp2KUQVPlnarPAw--
2GB.tmp : 0 MB in 1 s (128 kB/s), total 2048 MB
The default character set for F*EX is UTF-8, because most clients seem to
use it. The problem is, it is nowhere defined which character set HTTP
uses itself.
Important for programmers: The perl variables of the F*EX CGI's (fup, fop,
etc) have their UTF-8 flag turned off. This means, they contain UTF-8 data
in binary representation.
F*EX has support for many languanges. For every language a translated
version will be installed in $HOME/locale/
The switch is done at runtime via locale cookie (by fexsrv).
Some common functions from fex.pp must be present at any time for any
locale in any language, eg notification functions. Therfore there is
$HOME/lib/lf.pl (extracted functions from fex.pp).
I have additional authentication modules for RADIUS, LDAP, mailman and
POP. Please email me if you are interested.
framstag@rus.uni-stuttgart.de
|