1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489
|
2.1.1 (2024-04-08)
------------------
* `#267 <https://github.com/pytest-dev/execnet/issue/267>`__ Fixed regression
in 2.1.0 where the ``strconfig`` argument to ``load``/``loads`` is ignored.
2.1.0 (2024-04-05)
------------------
* `#243 <https://github.com/pytest-dev/execnet/pull/243>`__: Added ``main_thread_only``
execmodel which is derived from the thread execmodel and only executes ``remote_exec``
calls in the main thread.
Callers of ``remote_exec`` must use the returned channel to wait for a task to complete
before they call remote_exec again, otherwise the ``remote_exec`` call will fail with a
``concurrent remote_exec would cause deadlock`` error. The main_thread_only execmodel
provides solutions for `#96 <https://github.com/pytest-dev/execnet/issues/96>`__ and
`pytest-dev/pytest-xdist#620 <https://github.com/pytest-dev/pytest-xdist/issues/620>`__
(pending a new `pytest-xdist` release).
Also fixed ``init_popen_io`` to use ``closefd=False`` for shared stdin and stdout file
descriptors, preventing ``Bad file descriptor`` errors triggered by test_stdouterrin_setnull.
* The library is now typed and the typing is exposed to type-checkers.
* Re-exported ``Gateway``, ``Channel``, ``DumpError`` and ``LoadError`` from
``execnet``. The constructors are private.
* Fixed ``GatewayBase.join()`` timeout argument getting ignored.
* Removed support for Python 3.7.
* Added official support for Python 3.12.
2.0.2 (2023-07-09)
------------------
* Re-release without code changes, just to include ``tox.ini`` into the source distribution.
2.0.1 (2023-07-08)
------------------
* Re-release without code changes, just to include docs and tests into the source distribution.
2.0.0 (2023-07-06)
------------------
* Removed support for Python < 3.7.
- Applied ``pyupgrade --py37-plus``.
- Minimal ``mypy`` fixes and dropped Python 2 support code.
* Migrated packaging to ``hatch``.
* Dropped deprecated APIs of old makegateway names.
* Removed ``py`` testing dependency.
* Explicitly pass ``encoding`` when opening files in the gateway to get rid of warnings when using ``PYTHONWARNDEFAULTENCODING=1`` (#195).
* Fixed error when loading source code files from a path containing non-ascii characters.
1.9.0 (2021-06-13)
------------------
* Removed the ``apipkg`` dependency.
1.8.1 (2021-05-27)
------------------
* Update calls of `threading.Event.isSet()` to `threading.Event.is_set()`, which avoids a deprecation warning with Python 3.10.
1.8.0 (2021-01-27)
------------------
* Dropped support for Python 3.4.
* `#118 <https://github.com/pytest-dev/execnet/pull/118>`__: Fixed internal leak that should make
``execnet`` execute remote code in the main thread more often; previously it would sometimes
spawn a thread to execute a ``remote_exec`` call, even when the caller
didn't issue multiple ``remote_exec`` calls at the same time. Some frameworks require code
to execute in the main thread, so the previous behavior would break them on occasion (see
`pytest-dev/pytest-xdist#620 <https://github.com/pytest-dev/pytest-xdist/issues/620>`__
for an example).
* `#115 <https://github.com/pytest-dev/execnet/pull/115>`__: Current working directory is now
restored when calling ``script/socketserver.py``. The script now also loops by default
when called from the command-line.
1.7.1 (2019-08-28)
------------------
* `#108 <https://github.com/pytest-dev/execnet/issues/108>`__: Revert ``linecache`` optimization introduced in ``1.7.0`` which
broke remote execution.
1.7.0 (2019-08-08)
------------------
* `#102 <https://github.com/pytest-dev/execnet/pull/102>`__: Show paths in stack traces
generated by ``remote_exec()``.
* `#100 <https://github.com/pytest-dev/execnet/pull/100>`__: Fix flaky hangs in ``workerpool.waitall``.
1.6.1 (2019-07-22)
------------------
* `#98 <https://github.com/pytest-dev/execnet/pull/98>`__: Internal change to avoid
using deprecated ``funcargs`` name in pytest 5+.
1.6.0 (2019-03-31)
------------------
* ``execnet`` no longer supports Python 2.6 and 3.3 (#85). Users of those Python versions
using a recent enough ``pip`` should not be affected, as ``pip`` will only install
``1.5.0`` for them.
* Update test suite to support ``pytest>4``.
1.5.0 (2017-10-16)
------------------
- support shell escaping in python pathnames of popen.
Eugene Ciurana discovered that execnet breaks if you use
pathnames with spaces in a "python=" part of a spec.
We now use shlex.split to split the string. There is a
potential for regressions if you used quote or escape
sequences as part of your python command.
- Only insert importdir into sys.path if it is not already in the path.
This prevents a bug when using enum34 with python 3.6 and
pytest-xdist.
The issue is that enum34 installs an 'enum' module in site-packages
which is normally shadowed by the stdlib version of enum, however in
gateway_bootstrap.py site-packages is added at the front the the
search path. This means on the workers enum34 is hit for import enum
which in turn causes import re to fail (as it makes use of the new
enum features in 3.6).
- fix #49 - use inspect.getfullargspec if possible to avoid deprecationwarnings
- fix #56 - use partials in safe_terminate to avoid a bad carried binding
- fix spec parsing on Windows due to path containing '\' characters.
1.4.1 (2015-09-02)
------------------
- fix a regression of the Serializer created by the implied opcode ordering
which resulted in a incompatible opcode mapping
*warning* stored serialized objects created with 1.4.0 are incompatible
with previous versions and future versions
additionally stored serialized objects containing complex objects will
have a incompatible opcode when read with execnet < 1.4.0
and won't be loadable with execnet 1.4.0 either
its strongly suggested to avoid using the Serializer of execnet 1.4.0
this affects devpi and the external pytest-cache plugin
1.4
----
- de-vendor apipkg and use the pypi dependency instead
(this also fixes the bpython interaction issues)
- Fix issue38: provide ability to connect to Vagrant VMs easily
using :code:`vagrant_ssh=default` or :code:`vagrant_ssh=machinename`
this feature is experimental and will be refined in future releases.
Thanks Christian Theune for the discussion and the initial pull request.
- add support for serializing the "complex" type. Thanks Sebastian
Koslowski.
1.3
--------------------------------
- fix issue33: index.txt to correctly mention MIT instead of GPL.
- fix issue35: adapt some doctests, fix some channel tests for py3.
- use subprocess32 when available for python < 3.
- try to be a bit more careful when interpreter is shutting down
to avoid random exceptions, thanks Alfredo Deza.
- ignore errors on orphan file removal when rsyncing
- fix issue34: limit use of import based bootstrap
1.2
--------------------------------
- fix issue22 -- during interpreter shutdown don't throw
an exception when we can't send a termination sequence
anymore as we are about to die anyway.
- fix issue24 -- allow concurrent creation of gateways
by guarding automatic id creation by a look.
Thanks tlecomte.
- majorly refactor internal thread and IO handling.
execnet can now operate on different thread models,
defaults to "thread" but allows for eventlet and
gevent if it is installed.
- gateway.remote_exec() will now execute in multiple
threads on the other side by default. The previous
necessity of running "gateway.remote_init_threads()"
to allow for such concurrency is gone. The latter
method is now a no-op and will be removed in future
versions of execnet.
- fix issue20: prevent AttributError at interpreter shutdown
by not trying to send close/last_message messages if the
world around is half destroyed.
- fix issue21: allow to create local gateways with sudo aka
makegateway("popen//python=sudo python").
Thanks Alfredo Deza for the PR.
- streamline gateway termination and simplify proxy
implementation. add more internal tracing.
- if execution hangs in computation, we now try to
send a SIGINT to ourselves on Unix platforms
instead of just calling thread.interrupt_main()
- change license from GPL to MIT
- introduce execnet.dump/load variants of dumps/loads
serializing/unserializing mechanism.
- improve channel.receive() communication latency on python2
by changing the default timeout of the underlying Queue.get
to a regular None instead of the previous default -1
which caused an internal positive timeout value
(a hack probably introduced to allow CTRL-C to pass
through for <python2.5 versions).
- extended ssh-syntax to allow passing of command line args,
e.g. "ssh= -p 50 hostname". The options are passed to
the underlying ssh client binary. Thanks tundish.
- fix issue15: interoperability with inspect.getstack().
Thanks Peter Feiner.
- fix issue10 : skip PYTHONDONTWRITEBYTECODE test if
it we are running with PYTHONDONTWRITEBYTECODE set.
- dont try the jython pid fixup on a RemoteIO
- avoid accidentally setting exc_info() in gateway_base.py
1.1
--------------------------------
- introduce execnet.dumps/loads providing serialization between
python interpreters.
- group.remote_exec now supports kwargs as well
- support per channel string coercion configuration
- Popen2IO.read now reads correct amounts of bytes from nonblocking fd's
- added a ``dont_write_bytecode`` option to Popen gateways, this sets the
``sys.dont_write_bytecode`` flag on the spawned process, this only works on
CPython 2.6 and higher. Thanks to Alex Gaynor.
- added a pytest --broken-isp option to skip tests that assume
DNS queries for unknown hosts actually are resolved as such (Thanks
Alex Gaynor)
- fix issue 1 - decouple string coercion of channels and gateway
- fix issue #2 - properly reconfigure the channels string coercion for rsync,
so it can send from python2 to python3
- fix issue #9 - properly terminate the worker threadpools in safe_terminate
- fix issue #8 - no longer kill remote pids locally on jython ssh gateways
- refactor socketserver, so it can be directly remote_exec'd for starting a socket gateway on a remote
1.0.9
--------------------------------
- add gw.reconfigure() to configure per gateway options. Currently supported:
py2str_as_py3str and py3str_as_py2str to configure string deserialization
- channel.makefile() objects now have a isatty() returning False
- group.allocate_id(spec) allows to early-determine an (automatic) id
- internal refactorings and cleanups (thanks Ronny Pfannschmidt):
- refactor message types into received handler functions
- refactor b(chr(opcode)) to bchr(opcode)
- reorder Message ctor args, rename msgtype to msgcode
- refactor gateway.send to take message's init args instead of a message
- inline and remove Message.writeto/readfrom
- refactor collection loading to avoid the indirection over tuple
- remove the unused NamedThreadPool
1.0.8
--------------------------------
- new ``gateway.remote_exec(func, **kwargs)`` style for executing
a pure function with parameters. The function on the remote
side also needs to accept a ``channel`` which allows it to
communicate back and forth. Thanks to Ronny Pfannschmidt
for implementing it with special kudos to Maciej Fijalkowski
for writing a "pure-function" checker so that on Python2.6
onwards non-pure functions will be rejected.
- enhance rsyncing to also sync permissions (stat().st_mode)
of directories and files.
(should also resolve http://bitbucket.org/hpk42/py-trunk/issue/68/)
- fix rsyncing of symlinks, thanks to Charles Solar
(should also resolve http://bitbucket.org/hpk42/py-trunk/issue/70/)
- update internal usage of apipkg to 1.0b6
- remote_exec(module) now makes sure that the linecache is updated
before reading and sending the source. thanks Ronny, Matt.
- removed all trailing whitespace from source files
1.0.7
--------------------------------
- try to avoid a random KeyboardInterrupt Error when threads
are ending.
- extend xspec syntax to allow for one or multiple "env:NAME=value"
environment variable settings which will be set on the remote side.
(thanks Jakub Gustak)
1.0.6
--------------------------------
- fix jython/windows interactions
- fix waitclose/callback-with-endmarker race condition
- fix race condition where multiple threads sending data over channels
would crash the serializer and process
1.0.5
--------------------------------
- more care during receiver-thread finalization during interp-shutdown,
should get rid of annoying and meaningless exceptions
- fix glitch in ssh-fileserver example
- experimentally add "setup.py test" support - will run py.test
1.0.4
--------------------------------
- try to deal more cleanly with interpreter shutdown setting globals to
None. this avoids (hopefully) some bogus tracebacks at process exit.
1.0.3
--------------------------------
- refine termination some more: CTRL-C and gateway.exit will
now try harder to interrupt remote execution. this
helps to avoid left-over ssh-processes.
- fix read-on-non-blocking-files issue probably related to jython only:
the low-level read on subprocess pipes may be non-blocking, returning
less bytes than requested - so we now loop.
- Windows/python2.4: fix bug that killing subprocesses would fail
- make RemoteError and TimeoutError available directly on execnet namespace
- fix some doc and test issues (thanks thm and ronny), add ssh_fileserver example
- update internal copy of apipkg
- always skip remote tests if no ssh specs given
1.0.2
--------------------------------
- generalize channel-over-channel sending: you can now have channels
anywhere in a data structure (i.e. as an item of a container type).
Add according examples.
- automatically close a channel when a remote callback raises
an exception, makes communication more robust because until
now failing callbacks rendered the receiverthread unusable
leaving the remote side in-accessible.
- internally split socket gateways, speeds up popen-gateways
by 10% (now at <50 milliseconds per-gateway on a 1.5 GHZ machine)
- fix bug in channel.receive() that would wrongly raise a TimeoutError
after 1000 seconds (thanks Ronny Pfannschmidt)
1.0.1
--------------------------------
- revamp and better structure documentation
- new method: gateway.hasreceiver() returns True
if the gateway is still receive-active. remote_status
now only carries information about remote execution status.
- new: execnet.MultiChannel provides basic iteration/contain interface
- new: execnet.Group can be indexed by integer
- new: group.makegateway() uses group.default_spec if no spec is given
and the execnet.default_group uses ``popen`` as a default spec.
- have popen-gateways use imports instead of source-strings,
also improves debugging/tracebacks, as a side effect
popen-gateway startup can be substantially faster (>30%)
- refine internal gateway exit/termination procedure
and introduce group.terminate(timeout) which will
attempt to kill all subprocesses that did not terminate
within time.
- EOFError on channel.receive/waitclose if the other
side unexpectedly went away. When a gateway exits
it now internally sends an explicit termination message
instead of abruptly closing.
- introduce a timeout parameter to channel.receive()
and default to periodically internally wake up
to let KeyboardInterrupts pass through.
- EXECNET_DEBUG=2 will cause tracing to go to stderr,
which with popen worker gateways will relay back
tracing to the instantiator process.
1.0.0
--------------------------------
* introduce execnet.Group for managing gateway creation
and termination. Introduce execnet.default_group through which
all "global" calls are routed. cleanup gateway termination.
All Gateways get an id through which they can be
retrieved from a group object.
* deprecate execnet.XYZGateway in favour of direct makegateway() calls.
* refine socketserver-examples, experimentally introduce a
way to indirectly setup a socket server ("installvia")
through a gateway url.
* refine and automatically test documentation examples
1.0.0b3
--------------------------------
* fix EXECNET_DEBUG to work with win32
* add support for serializing longs, sets and frozensets (thanks
Benjamin Peterson)
* introduce remote_status() method which on the low level gives
information about the remote side of a gateway
* disallow explicit close in remote_exec situation
* perform some more detailed tracing with EXECNET_DEBUG
1.0.0b2
--------------------------------
* make internal protocols more robust against serialization failures
* fix a serialization bug with nested tuples containing empty tuples
(thanks to ronny for discovering it)
* setting the environment variable EXECNET_DEBUG will generate per
process trace-files for debugging
1.0.0b1
----------------------------
* added new examples for NumPy, Jython, IronPython
* improved documentation
* include apipkg.py for lazy-importing
* integrated new serializer code from Benjamin Peterson
* improved support for Jython-2.5.1
1.0.0alpha2
----------------------------
* improve documentation, new website
* use sphinx for documentation, added boilerplate files and setup.py
* fixes for standalone usage, adding boilerplate files
* imported py/execnet and made it work standalone
|