1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
|
Description: WIP dont-merge: Clean-up memcache connection sockets
The ConnectionPool currently opens sockets, but never closes them. As a
result, any client using oslo.cache memcache pool leaves sockets in a
CLOSE_WAIT state, with the source port not being re-usable.
.
In our production system, were we have A LOT of activity, this creates
a storm of non-reusable ports: all source ports are in use, making the
node unuseable for other things.
.
This patch adds a __del__ destructor closing inactive connections,
fixing the issue.
Author: Thomas Goirand <zigo@debian.org>
Date: Mon, 06 Nov 2023 10:55:45 +0100
Change-Id: I09d632346c76d1aff7c534f0d040162d1985f548
Forwarded: https://review.opendev.org/c/openstack/oslo.cache/+/900158
Last-Update: 2023-11-14
Index: python-oslo.cache/oslo_cache/_memcache_pool.py
===================================================================
--- python-oslo.cache.orig/oslo_cache/_memcache_pool.py
+++ python-oslo.cache/oslo_cache/_memcache_pool.py
@@ -114,6 +114,27 @@ class ConnectionPool(queue.Queue):
self._do_log(
LOG.warning, "Unable to cleanup a connection: %s", e)
+ def __del__(self):
+ """Delete the connection pool.
+
+ Destory all connections left in the queue.
+ """
+ while True:
+ # As per https://docs.python.org/3/library/collections.html
+ # self.queue.pop() will raise IndexError when no elements are
+ # present, ending the while True: loop.
+ # The logic loops over all connections in the queue but it does
+ # not retry for a single one in case a connection closure fails
+ # then it leaves that one and process the next.
+ try:
+ conn = self.queue.pop().connection
+ self._destroy_connection(conn)
+ except IndexError:
+ break
+ except Exception as e:
+ self._do_log(
+ LOG.warning, "Unable to cleanup a connection: %s", e)
+
def _create_connection(self):
"""Returns a connection instance.
|
