1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
|
=== Checklist for MDP release ===
- Check that new nodes have been explicitly imported in nodes/__init__.py and that they are listed in __all__:
# create a list of defined nodes with:
git grep 'class .*Node(' mdp/nodes | grep -v test | grep -v Scikits | cut -d ':' -f 2 | cut -d ' ' -f 2 | cut -d '(' -f 1 | sort > /tmp/list_defined
# create a list of nodes imported in mdp.nodes with:
python3 -c "import sys, mdp; [sys.stdout.write(i+'\n') for i in sorted([obj for obj in mdp.nodes.__dict__ if obj.endswith('Node') and not obj.endswith('ScikitsLearnNode')])]" > /tmp/list_in_dict
# create a list of nodes in __all__ with:
python3 -c "import sys, mdp; [sys.stdout.write(i+'\n') for i in sorted([obj for obj in mdp.nodes.__all__ if obj.endswith('Node') and not obj.endswith('ScikitsLearnNode')])]" > /tmp/list_in_all
# compare those listsm for example with
meld /tmp/list_defined /tmp/list_in_dict /tmp/list_in_all
# These lists should be almost identical. Some nodes defined or found in the dict could be used only internally,
# and thus not available through __all__, as of 2020.04.18 the only nodes that do not need to be in all three
# lists are:
- ICANode -> this is an internal node to be subclassed by nodes implementing some form of ICA
- ProjectionNode -> this is an internal node used by XSFA
- _ExpansionNode -> this is used internally by many SFA nodes)
- Make sure that mdp/__init__.py has the right version number
- Update date in variable __copyright__ in file mdp/__init__.py and in COPYRIGHT
- Update CHANGES: you can generate a new bunch of CHANGES with:
git log --no-color --pretty="format:%w(79,0,12)%ad: %s%+b" --date=short --no-merges --since=$LASTRELEASE
where LASTRELEASE is the tag of the last release, e.g. MDP-3.6
You can then prepend the output of this command to the original CHANGES file,
but *even better* would be to edit the result to only keep the changes
that are relevant for the user like incompatibilities, new features, etc..
Not every commit is interesting in CHANGES: the exact commits are available in git.
- Test that MDP can be installed with pip locally:
# create a wheel
python3 setup.py sdist bdist_wheel
# create a new Python3 virtualenv
python3 -m venv /tmp/mdp_test
# activate the virtualenv
source /tmp/mdp_test/bin/activate
# install the wheel in the virtualenv
pip install dist/XXX.whl
# this should also install MDP hard-dependencies (numpy, future)
# manually install MDP soft dependencies
pip install scipy scikit-learn joblib libsvm pytest
# change directory to be sure we are importing the pip-installed MDP
cd ~
# now we should be able to import
python -c 'import mdp; print(mdp.__file__, mdp.__version__)'
# -> in the above verify that the file is in the clone of the MDP repo,
# and that the version is correct
# now we can run the tests
python -c 'import mdp; mdp.test(); import bimdp; bimdp.test()'
# -> verify that all soft depenencies are correctly detected by MDP, for example
================================================== test session starts =============
platform linux -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
python: 3.8.2.final.0
mdp: 3.6
parallel python: NOT AVAILABLE: No module named 'pp'
shogun: NOT AVAILABLE: No module named 'shogun'
libsvm: 3.23.0.4
joblib: 0.14.1
sklearn: 0.22.2.post1
numx: scipy 1.4.1
symeig: scipy.linalg.eigh
Random Seed: 1343069334
rootdir: /tmp/mdp_test/lib/python3.8/site-packages/mdp/test
# -> also, of course check that all tests are passing
- create a release notes file (this is an edit version of the CHANGES file,
it will be used as annotation for the git tag, and will be shown on GitHub in the
release package)
- tag release in git and use the release notes file as annotation for the tag
git tag -a MDP-3.6 -F RELEASE_NOTES
- push the tag git push --tags
###### The stuff below is obsolete and needs to be updated ######
- generate tutorial, website, and API documentation [make website]
- change homepage colors
- short/long description should go: on SF.net description, tutorial,
home page, modules __init__, software.incf.net.
- update the package on PyPI:
# first test on the test PyPI instance (you have to register there first)
# - create a ~/.pypirc first:
cat > ~/.pypirc <<EOF
[distutils]
index-servers=
test
[test]
repository = https://testpypi.python.org/pypi
username = otizonaizit
EOF
# - register
python3 setup.py register -r test
# go and check if it is there!
# now use twine to upload and sign the packages
twine upload -r test -s dist/MDP-3.4*
# now create another virtualenv and try to test installation
virtualenv mdp-27 && source /home/tiziano/mdp-27/bin/activate
pip install -i https://testpypi.python.org/pypi MDP
# if it fails because future or numpy are not on the test pypi server,
# just install them with pip install without -i
#
# If all of the above works, you can finally upload to the real PyPI:
# modify your ~/.pypirc:
cat > ~/.pypirc <<EOF
[distutils]
index-servers=
pypi
test
[test]
repository = https://testpypi.python.org/pypi
username = otizonaizit
[pypi]
repository = https://pypi.python.org/pypi
username = otizonaizit
EOF
# now register
python3 setup register
# upload to pipy
twine upload -s dist/MDP-3.4*
# test that it works with pip
virtualenv mdp-27 && source /home/tiziano/mdp-27/bin/activate
pip install mdp
# yeah, it works!!!
- update on SF.net:
release files:
- sftp username@frs.sourceforge.net
- cd /home/frs/project/mdp-toolkit/mdp-toolkit/
- create a new directory for the release, for example for release 3.0:
mkdir 3.0
cd 3.0
- upload the files there (note: the release notes should be named
README.txt):
file to upload are: .tar.gz, .zip, .exe, tutorial, release notes file
- login to sourceforge, go to "Files"
- select the new created directory
- select the wheel file and set it as a default for windows and
Mac by clicking on the "i" icon on the right,
- select the tar.gz as a default for everything else
- at that point the readme file should be automatically shown as release
note file
if README.txt is not shown, delete it and upload it through the web interface.
make sure that it is shown.
- more info: https://sourceforge.net/apps/trac/sourceforge/wiki/Release%20files%20for%20download
- make the website within a clone of the docs repository with:
- make website
- be careful to read all warnings and messages, often things do not work as
expected.
- upload the pdf tutorial, which is in build/latex/MDP-tutorial.pdf, to sf.net
as explained above for the source tarballs.
- synchronize the site with:
cd build/html
rsync -av --delete-after . username@web.sourceforge.net:/home/project-web/mdp-toolkit/htdocs/
- more info: http://alexandria.wiki.sourceforge.net/Project+Web,+Shell,+VHOST+and+Database+Services
- tag the docs repository:
git tag -a MDP-3.5
git push --tags
After release:
- update version number in __init__
- send announcement to:
connectionists: connectionists@cs.cmu.edu
ML-news: ML-news@googlegroups.com
numpy-discussion: numpy-discussion@scipy.org
Scipy users: scipy-user@scipy.org
mdp-users: mdp-toolkit@python.org
Python-announce: python-announce-list@python.org
- celebrate!!
|
