1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
|
..
This file is part of khmer, https://github.com/dib-lab/khmer/, and is
Copyright (C) 2012-2015 Michigan State University
Copyright (C) 2015 The Regents of the University of California.
It is licensed under the three-clause BSD license; see LICENSE.
Contact: khmer-project@idyll.org
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:
* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following
disclaimer in the documentation and/or other materials provided
with the distribution.
* Neither the name of the Michigan State University nor the names
of its contributors may be used to endorse or promote products
derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Contact: khmer-project@idyll.org
Development miscellany
======================
Third-party use
---------------
We ask that third parties who build upon the codebase to do so from a
versioned release. This will help them determine when bug fixes apply and
generally make it easier to collaborate. If more intensive modifications happen
then we request that the repository is forked, again preferably from a version
tag.
Build framework
---------------
`make` should build everything, including tests and "development" code.
git and GitHub strategies
-------------------------
Still in the works, but read `this
<http://scottchacon.com/2011/08/31/github-flow.html>`__.
Make a branch on dib-lab (preferred so others can contribute) or fork the
repository and make a branch there.
Each piece or fix you are working on should have its own branch; make a pull-
request to dib-lab/master to aid in code review, testing, and feedback.
If you want your code integrated then it needs to be mergable
Example pull request update using the command line:
#. Clone the source of the pull request (if needed)
``git clone git@github.com:mr-c/khmer.git``
#. Checkout the source branch of the pull request
``git checkout my-pull-request``
#. Pull in the destination of the pull request and resolve any conflicts
``git pull git@github.com:dib-lab/khmer.git master``
#. Push your update to the source of the pull request ``git push``
#. Jenkins will automatically attempt to build and test your pull requests.
Code coverage
-------------
Jenkins calculates code coverage for every build. Navigate to the results from
the master node first to view the coverage information.
Code coverage should never go down and new functionality needs to be tested.
Pipelines
---------
All khmer scripts used by a published recommended analysis pipeline must be
included in ``scripts/`` and meet the standards therein implied.
Command line scripts
--------------------
Python command-line scripts should use '-' instead of '_' in the name.
(Only filenames containing code for import should use _.)
Please follow the command-line conventions used under scripts/. This
includes most especially standardization of '-x' to be hash table size,
'-N' to be number of hash tables, and '-k' to always refer to the
k-mer size.
Command line thoughts:
If a filename is required, typically UNIX commands don't use a flag to
specify it.
Also, positional arguments typically aren't used with multiple files.
CTB's overall philosophy is that new files, with new names, should
be created as the result of filtering etc.; this allows easy
chaining of commands. We're thinking about how best to allow
override of this, e.g. ::
filter-abund.py <ct file> <filename> [ -o <filename.keep> ]
----
All code in ``scripts/`` must have automated tests; see
``tests/test_scripts.py``. Otherwise it belongs in ``sandbox/``.
When files are overwritten, they should only be opened to be overwritten
after the input files have been shown to exist. That prevents stupid
command line mistakes from trashing important files.
A general error should be signaled by exit code `1` and success by `0`. Linux
supports exit codes from `0` to `255` where the value `1` means a general
error. An exit code of `-1` will get converted to `255`.
----
CLI reading:
http://stackoverflow.com/questions/1183876/what-are-the-best-practices-for-implementing-a-cli-tool-in-perl
http://catb.org/esr/writings/taoup/html/ch11s06.html
http://figshare.com/articles/tutorial_pdf/643388
Python / C integration
----------------------
The Python extension that wraps the C++ core of khmer lives in
``khmer/_khmermodule.cc``
This wrapper code is tedious and annoying so we use a static analysis tool to
check for correctness.
https://gcc-python-plugin.readthedocs.org/en/latest/cpychecker.html
Developers using Ubuntu Precise will want to install the gcc-4.6-plugin-dev
package
Example usage: ::
CC="/home/mcrusoe/src/gcc-plugin-python/gcc-python-plugin/gcc-with-cpychecker
--maxtrans=512" python setup.py build_ext 2>&1 | less
False positives abound: ignore errors about the C++ standard library. This tool
is primarily useful for reference count checking, error-handling checking, and
format string checking.
Errors to ignore: "Unhandled Python exception raised calling 'execute' method",
"AttributeError: 'NoneType' object has no attribute 'file'"
Warnings to address: ::
khmer/_khmermodule.cc:3109:1: note: this function is too complicated
for the reference-count checker to fully analyze: not all paths were
analyzed
Adjust --maxtrans and re-run. ::
khmer/_khmermodule.cc:2191:61: warning: Mismatching type in call to
Py_BuildValue with format code "i" [enabled by default]
argument 2 ("D.68937") had type
"long long unsigned int"
but was expecting
"int"
for format code "i"
See below for a format string cheat sheet One also benefits by matching C type
with the function signature used later.
"I" for unsigned int
"K" for unsigned long long a.k.a khmer::HashIntoType.
|
