1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
|
.. _contributing:
==============
Contributing
==============
Development of ``sqlite-utils`` takes place in the `sqlite-utils GitHub repository <https://github.com/simonw/sqlite-utils>`__.
All improvements to the software should start with an issue. Read `How I build a feature <https://simonwillison.net/2022/Jan/12/how-i-build-a-feature/>`__ for a detailed description of the recommended process for building bug fixes or enhancements.
.. _contributing_checkout:
Obtaining the code
==================
To work on this library locally, first checkout the code. Then create a new virtual environment::
git clone git@github.com:simonw/sqlite-utils
cd sqlite-utils
python3 -mvenv venv
source venv/bin/activate
Or if you are using ``pipenv``::
pipenv shell
Within the virtual environment running ``sqlite-utils`` should run your locally editable version of the tool. You can use ``which sqlite-utils`` to confirm that you are running the version that lives in your virtual environment.
.. _contributing_tests:
Running the tests
=================
To install the dependencies and test dependencies::
pip install -e '.[test]'
To run the tests::
pytest
.. _contributing_docs:
Building the documentation
==========================
To build the documentation, first install the documentation dependencies::
pip install -e '.[docs]'
Then run ``make livehtml`` from the ``docs/`` directory to start a server on port 8000 that will serve the documentation and live-reload any time you make an edit to a ``.rst`` file::
cd docs
make livehtml
The `cog <https://github.com/nedbat/cog>`__ tool is used to maintain portions of the documentation. You can run it like so::
cog -r docs/*.rst
.. _contributing_linting:
Linting and formatting
======================
``sqlite-utils`` uses `Black <https://black.readthedocs.io/>`__ for code formatting, and `flake8 <https://flake8.pycqa.org/>`__ and `mypy <https://mypy.readthedocs.io/>`__ for linting and type checking.
Black is installed as part of ``pip install -e '.[test]'`` - you can then format your code by running it in the root of the project::
black .
To install ``mypy`` and ``flake8`` run the following::
pip install -e '.[flake8,mypy]'
Both commands can then be run in the root of the project like this::
flake8
mypy sqlite_utils
All three of these tools are run by our CI mechanism against every commit and pull request.
.. _contributing_just:
Using Just and pipenv
=====================
If you install `Just <https://github.com/casey/just>`__ and `pipenv <https://pipenv.pypa.io/>`__ you can use them to manage your local development environment.
To create a virtual environment and install all development dependencies, run::
cd sqlite-utils
just init
To run all of the tests and linters::
just
To run tests, or run a specific test module or test by name::
just test # All tests
just test tests/test_cli_memory.py # Just this module
just test -k test_memory_no_detect_types # Just this test
To run just the linters::
just lint
To apply Black to your code::
just black
To update documentation using Cog::
just cog
To run the live documentation server (this will run Cog first)::
just docs
And to list all available commands::
just -l
.. _release_process:
Release process
===============
Releases are performed using tags. When a new release is published on GitHub, a `GitHub Actions workflow <https://github.com/simonw/sqlite-utils/blob/main/.github/workflows/publish.yml>`__ will perform the following:
* Run the unit tests against all supported Python versions. If the tests pass...
* Build a wheel bundle of the underlying Python source code
* Push that new wheel up to PyPI: https://pypi.org/project/sqlite-utils/
To deploy new releases you will need to have push access to the GitHub repository.
``sqlite-utils`` follows `Semantic Versioning <https://semver.org/>`__::
major.minor.patch
We increment ``major`` for backwards-incompatible releases.
We increment ``minor`` for new features.
We increment ``patch`` for bugfix releass.
To release a new version, first create a commit that updates the version number in ``setup.py`` and the :ref:`the changelog <changelog>` with highlights of the new version. An example `commit can be seen here <https://github.com/simonw/sqlite-utils/commit/b491f22d817836829965516983a3f4c3c72c05fc>`__::
# Update changelog
git commit -m " Release 3.29
Refs #423, #458, #467, #469, #470, #471, #472, #475" -a
git push
Referencing the issues that are part of the release in the commit message ensures the name of the release shows up on those issue pages, e.g. `here <https://github.com/simonw/sqlite-utils/issues/458#ref-commit-b491f22>`__.
You can generate the list of issue references for a specific release by copying and pasting text from the release notes or GitHub changes-since-last-release view into this `Extract issue numbers from pasted text <https://observablehq.com/@simonw/extract-issue-numbers-from-pasted-text>`__ tool.
To create the tag for the release, create `a new release <https://github.com/simonw/sqlite-utils/releases/new>`__ on GitHub matching the new version number. You can convert the release notes to Markdown by copying and pasting the rendered HTML into this `Paste to Markdown tool <https://euangoddard.github.io/clipboard2markdown/>`__.
|
