File: HACKING

package info (click to toggle)
coquelicot 0.9.6-1.1
  • links: PTS, VCS
  • area: main
  • in suites: sid
  • size: 1,152 kB
  • sloc: ruby: 4,327; sh: 70; makefile: 69
file content (258 lines) | stat: -rw-r--r-- 8,223 bytes parent folder | download | duplicates (2)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
Development notes
=================

[Coquelicot] is written in Ruby and should be quite easy to improve for
anyone a little bit familiar with the Sinatra web framework. It is
mostly written using Behaviour Driven Development, making the test suite
a fine net to hack in confidence. So please go ahead!

[Coquelicot]: https://coquelicot.potager.org/

Setup a work environment
------------------------

As Coquelicot uses Bundle, the first step to work on Coquelicot
after cloning its Git repository is installing the proper dependencies by
issuing:

    bundle install

Basic operations
----------------

Coquelicot test suite is written using RSpec. Running the test suite is
just a matter of typing:

    bundle exec rspec

Running a test server can be done with:

    bundle exec coquelicot start --no-daemon

To update the translation source files, use:

    bundle exec rake gettext:po:update

This will update `po/coquelicot.pot` and merge the new strings in the various
`po/*/coquelicot.po` files.

Authentication mechanisms
-------------------------

The authentication part of Coquelicot has been made modular. Adding a
new authentication mechanism should be fairly straightforward.

A new authentication mechanism needs to provide the following 3 files,
with the following responsabilities:

 * `lib/coquelicot/auth/<METHOD>.rb`:

   A class implementing the actual authentication. This class must
   implement an `authenticate` method. It will receive the form fields
   as usual (params). This method must return true if upload
   should be allowed.

 * `public/javascripts/coquelicot.auth.<METHOD>.js:`

    This file must define 'authentication' as an object with the
    following methods:

    - `getData()`: return an object of all the necessary data
      to authenticate on the app side. Keys must have the same name
      as the input fields used to authenticate without JavaScript.
    - `focus()`: set the focus on the first authentication form field.
    - (optional) `handleSuccess()`: arbitrary action upon successful
      authentication. This is called after the livebox with
      authentication fields is closed.
    - (optional) `handleReject()`: arbitrary action when access
      is denied. One can reset authentication fields after a failed
      authentication.
    - (optional) `handleFailure()`: arbitrary action when there was
      a problem in the authentication procedure.

 * `views/auth/<METHOD>.haml`:

   A template with the necessary form fields that will be used for
   authentication.

The authentication mechanism is set in the configuration file and
can include options specific to the method chosen.

Implementation details
----------------------

Common application code lies in `Coquelicot::Application`, except for
one specific (and important) type of requests, namely `POST /update`.
These requests are handled directly at bare Rack level by
`Coquelicot::Rack::Upload`.

This allows to work directly with POST data as the browser is sending
it, so we can directly stream the uploaded file to our encrypted
on-disk containers.

The POST data must be in a very specific order, as we need to handle
authentication and other option fields before we start recording the
file content. Thanks to the W3C, the [HTML specification] states that
parts of the POST data must be delivered in the same order as the
controls appear in the `<form/>` container.

`Coquelicot::Rack::Multipart` exposes a simple DSL to parse the fields
as they are delivered. The later is used by `Coquelicot::Rack::Upload`
to perform its logic pretty nicely.

[HTML specification]: http://www.w3.org/TR/html4/interact/forms.html

Watch for buffered inputs!
--------------------------

Coquelicot is written in Ruby using Sinatra. Sinatra is based on the
Rack webserver interface. Rack specification mandates that applications
must be able to seek and rewind freely in the request content.

Request data is always received as a stream through the network. So in
order to comply with the specification, webservers implementing Rack
either buffer the input in memory (Webrick) or in a temporary file
(Thin, Passenger or Mongrel).

On top of that, when parsing `multipart/form-data` POST content,
`Rack::Request` (used by Sinatra) creates a new temporary file for
each files in the POST request.

For the specific needs of Coquelicot, these behaviours prevent users
from uploading large files (if `/tmp` is in memory) or breach their
privacy by writing a clear text version to disk.

To overcome these limitations, Coquelicot first uses a specific feature
of the Rainbows! webserver of streaming its input directly to
applications, and second bypasses `Rack::Request` to directly handle
POST content. Usage of any other Rack webserver is strongly discouraged
and should be restricted to development and testing.

Storage details
---------------

Files are stored in the directory specified by the 'depot_path' setting.
One file in Coquelicot is actually stored in two files: one for metadata and
one for the file content.

### Metadata file

The format is the following:

    --- 
    Coquelicot: "2.0"
    Salt: <8 bytes stored as Base64>
    Expire-at: <expiration time in seconds since epoch>
    --- 
    <encrypted metadata>

Encryption is done using OpenSSL. Cipher is AES-256-CBC with key and IV
created using the `pbkdf2_hmac_sha1()` implementation of PKCS5. The later
is fed using the former *Salt* and the given passphrase, using 2000
iterations.

Once decrypted, the metadata have the following format:

    --- 
    Created-at: <upload time in seconds since epoch>
    Filename: "<original file name>"
    Content-Type: "<MIME type>"
    Length: <content length is bytes>
    One-time-only: <true|false>

Headers must be valid YAML.

### Content file

The content file contains the stored file in encrypted form. Encryption is done
with the same algorithm and keys as the encrypted metadata (see above).

The file name of the content file is the same as the one for metada, with an
added suffix of '.content'. For example, if the metadata file name is
`mqeb4pfcru2ymq3e6se7`, the associated content file will be
`mqeb4pfcru2ymq3e6se7.content`.

### Expired files

Both the content file and the metadata file are truncated to zero length when
they are "expired".

### URL mapping

In order to map download URLs to file name, a simple text file ".links"
is used. It contains a line for each file in the form:

    <URL name> <metadata file name>

### Changes history

  version 2.0
  :    Current version described above.

  version 1.0
  :    File content is in the same file as the metadata. Content is put in the
       after the metadata and an extra "--- \n".

Sending patches
---------------

Please send patches to the users and developers [mailing list]. They are best
prepared using `git format-patch`.

[mailing list]: https://listes.potager.org/listinfo/coquelicot

How to make a new release?
--------------------------

 1. Bump version number in `lib/coquelicot/version.rb` and `Gemfile.lock`.
    Don't forget to commit the changes.

 2. Add a new entry in the NEWS file. For an outline:

        git log --reverse --oneline $(git describe --abbrev=0)..

    Don't forget to commit the changes.

 3. Tag the release:

        git tag -s coquelicot-$VERSION -m "coquelicot $VERSION"

 4. Push changes to the main repository:

        git push origin master coquelicot-$VERSION

 5. Create a source tarball:

        bundle exec rake create_archive

 6. Sign it:

        gpg --armor --detach-sign coquelicot-$VERSION.tar.gz

 7. Switch to the website:

        cd ../website

 8. Move the source tarball and signature to the website:

        mv ../git/coquelicot-$VERSION.tar.gz* static/dist/

 9. Add them to the website repository:

        git add static/dist/coquelicot-$VERSION.tar.gz*

 10. Update the version on the website homepage:

         sed -e "s/coquelicot-$PREVIOUS_VERSION/coquelicot-$VERSION/g" \
             -i dynamic/index.md

 11. Commit changes to the website.

 12. Push the updated website:

         make push
         git push origin master

 13. Announce the release on `coquelicot@potager.org` mailing-list.

 14. Announce the release on `freecode.com`.