1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458
|
from __future__ import annotations
import abc
import ipaddress
import json
import queue
import re
import threading
import time
import urllib.parse
from collections import defaultdict
from collections.abc import Iterable
from collections.abc import Mapping
from collections.abc import MutableMapping
from contextlib import contextmanager
from contextlib import suppress
from copy import copy
from enum import Enum
from re import Pattern
from typing import TYPE_CHECKING
from typing import Any
from typing import Callable
from typing import ClassVar
from typing import Optional
from typing import Union
import werkzeug.http
from werkzeug import Request
from werkzeug import Response
from werkzeug.datastructures import Authorization
from werkzeug.datastructures import MultiDict
from werkzeug.serving import make_server
if TYPE_CHECKING:
from ssl import SSLContext
from types import TracebackType
from werkzeug.serving import BaseWSGIServer
URI_DEFAULT = ""
METHOD_ALL = "__ALL"
HEADERS_T = Union[
Mapping[str, Union[str, Iterable[str]]],
Iterable[tuple[str, str]],
]
HVMATCHER_T = Callable[[str, Optional[str], str], bool]
class Undefined:
def __repr__(self):
return "<UNDEFINED>"
UNDEFINED = Undefined()
class Error(Exception):
"""
Base class for all exception defined in this package.
"""
class NoHandlerError(Error):
"""
Raised when a :py:class:`RequestHandler` has no registered method to serve the request.
"""
class HTTPServerError(Error):
"""
Raised when there's a problem with HTTP server.
"""
class NoMethodFoundForMatchingHeaderValueError(Error):
"""
Raised when a :py:class:`HeaderValueMatcher` has no registered method to match the header value.
"""
class WaitingSettings:
"""Class for providing default settings and storing them in HTTPServer
:param raise_assertions: whether raise assertions on unexpected request or timeout or not
:param stop_on_nohandler: whether stop on unexpected request or not
:param timeout: time (in seconds) until time is out
"""
def __init__(
self,
raise_assertions: bool = True, # noqa: FBT001
stop_on_nohandler: bool = True, # noqa: FBT001
timeout: float = 5,
):
self.raise_assertions = raise_assertions
self.stop_on_nohandler = stop_on_nohandler
self.timeout = timeout
class Waiting:
"""Class for HTTPServer.wait context manager
This class should not be instantiated directly."""
def __init__(self):
self._result = None
self._start = time.monotonic()
self._stop = None
def complete(self, result: bool): # noqa: FBT001
self._result = result
self._stop = time.monotonic()
@property
def result(self) -> bool:
return bool(self._result)
@property
def elapsed_time(self) -> float:
"""Elapsed time in seconds"""
if self._stop is None:
raise TypeError("unsupported operand type(s) for -: 'NoneType' and 'float'")
return self._stop - self._start
class HeaderValueMatcher:
"""
Matcher object for the header value of incoming request.
:param matchers: mapping from header name to comparator function that accepts actual and expected header values
and return whether they are equal as bool.
"""
DEFAULT_MATCHERS: ClassVar[MutableMapping[str, Callable[[str | None, str], bool]]] = {}
def __init__(self, matchers: Mapping[str, Callable[[str | None, str], bool]] | None = None):
self.matchers = self.DEFAULT_MATCHERS if matchers is None else matchers
@staticmethod
def authorization_header_value_matcher(actual: str | None, expected: str) -> bool:
func = getattr(Authorization, "from_header", None)
if func is None: # Werkzeug < 2.3.0
func = werkzeug.http.parse_authorization_header # type: ignore[attr-defined]
return func(actual) == func(expected) # type: ignore
@staticmethod
def default_header_value_matcher(actual: str | None, expected: str) -> bool:
return actual == expected
def __call__(self, header_name: str, actual: str | None, expected: str) -> bool:
try:
matcher = self.matchers[header_name]
except KeyError:
raise NoMethodFoundForMatchingHeaderValueError(
"No method found for matching header value: {}".format(header_name)
)
return matcher(actual, expected)
HeaderValueMatcher.DEFAULT_MATCHERS = defaultdict(
lambda: HeaderValueMatcher.default_header_value_matcher,
{"Authorization": HeaderValueMatcher.authorization_header_value_matcher},
)
class QueryMatcher(abc.ABC):
"""
Abstract class for QueryMatchers
get_comparing_values should return a 2-element tuple whose elements will
be compared.
"""
def match(self, request_query_string: bytes) -> bool:
values = self.get_comparing_values(request_query_string)
return values[0] == values[1]
@abc.abstractmethod
def get_comparing_values(self, request_query_string: bytes) -> tuple[Any, Any]:
pass
class StringQueryMatcher(QueryMatcher):
"""
Matches a query for a string or bytes specified
"""
def __init__(self, query_string: bytes | str):
"""
:param query_string: the query string will be compared to this string or bytes.
If string is specified, it will be encoded by the encode() method.
The query must not start with '?' but will be exactly (byte-by-byte) equal
the actual query string of the incoming request.
"""
if not isinstance(query_string, (str, bytes)):
raise TypeError("query_string must be a string, or a bytes-like object")
self.query_string = query_string
def get_comparing_values(self, request_query_string: bytes) -> tuple[bytes, bytes]:
if isinstance(self.query_string, str):
query_string = self.query_string.encode()
elif isinstance(self.query_string, bytes): # type: ignore
query_string = self.query_string
else:
raise TypeError("query_string must be a string, or a bytes-like object")
return (request_query_string, query_string)
class MappingQueryMatcher(QueryMatcher):
"""
Matches a query string to a dictionary or MultiDict specified
"""
def __init__(self, query_dict: Mapping[str, str] | MultiDict[str, str]):
"""
:param query_dict: if dictionary (Mapping) is specified, it will be used as a
key-value mapping where both key and value should be string. If there are multiple
values specified for the same key in the request, the first element will be used.
If you want to match multiple values, use a MultiDict object from werkzeug, which
represents multiple values for one key.
"""
self.query_dict = query_dict
def get_comparing_values(self, request_query_string: bytes) -> tuple[Mapping[str, str], Mapping[str, str]]:
query = MultiDict(urllib.parse.parse_qsl(request_query_string.decode("utf-8")))
if isinstance(self.query_dict, MultiDict):
return (query, self.query_dict)
else:
return (query.to_dict(), dict(self.query_dict))
class BooleanQueryMatcher(QueryMatcher):
"""
Matches the query depending on the boolean value
"""
def __init__(self, result: bool): # noqa: FBT001
"""
:param result: if this parameter is true, the query match will be always
successful. Otherwise, no query match will be successful.
"""
self.result = result
def get_comparing_values(self, request_query_string: bytes): # noqa: ARG002
if self.result:
return (True, True)
else:
return (True, False)
def _create_query_matcher(query_string: None | QueryMatcher | str | bytes | Mapping[str, str]) -> QueryMatcher:
if isinstance(query_string, QueryMatcher):
return query_string
if query_string is None:
return BooleanQueryMatcher(result=True)
if isinstance(query_string, (str, bytes)):
return StringQueryMatcher(query_string)
if isinstance(query_string, Mapping):
return MappingQueryMatcher(query_string)
raise TypeError("Unable to cast this type to QueryMatcher: {!r}".format(type(query_string)))
class URIPattern(abc.ABC):
@abc.abstractmethod
def match(self, uri: str) -> bool:
"""
Matches the provided URI.
:param uri: URI of the request. This is an absolute path starting
with "/" and does not contain the query part.
:return: True if there's a match, False otherwise
"""
class RequestMatcher:
"""
Matcher object for the incoming request.
It defines various parameters to match the incoming request.
:param uri: URI of the request. This must be an absolute path starting with ``/``, a
:py:class:`URIPattern` object, or a regular expression compiled by :py:func:`re.compile`.
:param method: HTTP method of the request. If not specified (or `METHOD_ALL`
specified), all HTTP requests will match.
:param data: payload of the HTTP request. This could be a string (utf-8 encoded
by default, see `data_encoding`) or a bytes object.
:param data_encoding: the encoding used for data parameter if data is a string.
:param headers: dictionary of the headers of the request to be matched
:param query_string: the http query string, after ``?``, such as ``username=user``.
If string is specified it will be encoded to bytes with the encode method of
the string. If dict is specified, it will be matched to the ``key=value`` pairs
specified in the request. If multiple values specified for a given key, the first
value will be used. If multiple values needed to be handled, use ``MultiDict``
object from werkzeug.
:param header_value_matcher: :py:class:`HeaderValueMatcher` that matches
values of headers, or a ``Callable[[str, Optional[str], str], bool]``
receiving the header key (from `headers`), header value (or `None`) and the expected
value (from `headers`) and should return ``True`` if the header matches, ``False`` otherwise.
:param json: a python object (eg. a dict) whose value will be compared to the request body after it
is loaded as json. If load fails, this matcher will be failed also. *Content-Type* is not checked.
If that's desired, add it to the headers parameter.
"""
def __init__(
self,
uri: str | URIPattern | Pattern[str],
method: str = METHOD_ALL,
data: str | bytes | None = None,
data_encoding: str = "utf-8",
headers: Mapping[str, str] | None = None,
query_string: None | QueryMatcher | str | bytes | Mapping[str, str] = None,
header_value_matcher: HVMATCHER_T | None = None,
json: Any = UNDEFINED,
):
if json is not UNDEFINED and data is not None:
raise ValueError("data and json parameters are mutually exclusive")
self.uri = uri
self.method = method
self.query_string = query_string
self.query_matcher = _create_query_matcher(self.query_string)
self.json = json
self.headers: Mapping[str, str] = {}
if headers is not None:
self.headers = headers
if isinstance(data, str):
data = data.encode(data_encoding)
self.data = data
self.data_encoding = data_encoding
self.header_value_matcher: HVMATCHER_T = HeaderValueMatcher()
if header_value_matcher is not None:
self.header_value_matcher = header_value_matcher
def __repr__(self):
"""
Returns the string representation of the object, with the known parameters.
"""
class_name = self.__class__.__name__
retval = "<{} ".format(class_name)
retval += (
"uri={uri!r} method={method!r} query_string={query_string!r} "
"headers={headers!r} data={data!r} json={json!r}>"
).format_map(self.__dict__)
return retval
def match_data(self, request: Request) -> bool:
"""
Matches the data part of the request
:param request: the HTTP request
:return: `True` when the data is matched or no matching is required. `False` otherwise.
"""
if self.data is None:
return True
return request.data == self.data
def match_uri(self, request: Request) -> bool:
path = request.path
if isinstance(self.uri, URIPattern):
return self.uri.match(path)
# this is python version depending
# in python 3.7 and above: it is re.Pattern
# below python 3.7 it is _sre.SRE_Pattern which cannot be accessed directly
elif isinstance(self.uri, re.compile("").__class__):
return bool(self.uri.match(path))
else:
# there could be a guard isinstance(self.uri, str) been here
# but we want to allow any object which provides the __eq__ parameter
# (note: in this case it will be not typeing correct)
#
# also, python will raise TypeError when self.uri is a conflicting type
return self.uri in (URI_DEFAULT, path)
def match_json(self, request: Request) -> bool:
"""
Matches the request data as json.
Load the request data as json and compare it to self.json which is a
json-serializable data structure (eg. a dict or list).
:param request: the HTTP request
:return: `True` when the data is matched or no matching is required. `False` otherwise.
"""
if self.json is UNDEFINED:
return True
try:
# do the decoding here as python 3.5 requires string and does not
# accept bytes
json_received = json.loads(request.data.decode(self.data_encoding))
except json.JSONDecodeError:
return False
except UnicodeDecodeError:
return False
return json_received == self.json
def difference(self, request: Request) -> list[tuple[str, str, str | URIPattern]]:
"""
Calculates the difference between the matcher and the request.
Returns a list of fields where there's a difference between the request and the matcher.
The returned list may have zero or more elements, each element is a three-element tuple
containing the field name, the request value, and the matcher value.
If zero-length list is returned, this means that there's no difference, so the request
matches the fields set in the matcher object.
"""
retval: list[tuple[str, Any, Any]] = []
if not self.match_uri(request):
retval.append(("uri", request.path, self.uri))
if self.method not in (METHOD_ALL, request.method):
retval.append(("method", request.method, self.method))
if not self.query_matcher.match(request.query_string):
retval.append(("query_string", request.query_string, self.query_string))
request_headers: dict[str, str | None] = {}
expected_headers: dict[str, str] = {}
for key, value in self.headers.items():
if not self.header_value_matcher(key, request.headers.get(key), value):
request_headers[key] = request.headers.get(key)
expected_headers[key] = value
if request_headers and expected_headers:
retval.append(("headers", request_headers, expected_headers))
if not self.match_data(request):
retval.append(("data", request.data, self.data))
if not self.match_json(request):
retval.append(("json", request.data, self.json))
return retval
def match(self, request: Request) -> bool:
"""
Returns whether the request matches the parameters set in the matcher
object or not. `True` value is returned when it matches, `False` otherwise.
"""
difference = self.difference(request)
return not difference
class RequestHandlerBase(abc.ABC):
"""
Represents a :py:class:`RequestHandler` object providing a response for the corresponding request.
"""
def respond_with_json(
self,
response_json: Any,
status: int = 200,
headers: Mapping[str, str] | None = None,
content_type: str = "application/json",
):
"""
Prepares a response with a serialized JSON object.
:param response_json: a JSON-serializable python object
:param status: the HTTP status of the response
:param headers: the HTTP headers to be sent (excluding the Content-Type header)
:param content_type: the content type header to be sent
"""
response_data = json.dumps(response_json, indent=4)
self.respond_with_data(response_data, status, headers, content_type=content_type)
def respond_with_data(
self,
response_data: str | bytes = "",
status: int = 200,
headers: HEADERS_T | None = None,
mimetype: str | None = None,
content_type: str | None = None,
):
"""
Prepares a response with raw data.
For detailed description please see the :py:class:`werkzeug.Response` object as the
parameters are analogue.
:param response_data: a string or bytes object representing the body of the response
:param status: the HTTP status of the response
:param headers: the HTTP headers to be sent (excluding the Content-Type header)
:param content_type: the content type header to be sent
:param mimetype: the mime type of the request
"""
self.respond_with_response(Response(response_data, status, headers, mimetype, content_type))
@abc.abstractmethod
def respond_with_response(self, response: Response):
"""
Prepares a response with the specified response object.
:param response: the response object which will be responded
"""
class RequestHandler(RequestHandlerBase):
"""
Represents a response function and a :py:class:`RequestHandler` object.
This class connects the matcher object with the function responsible for the response.
The respond handler function can be registered with the `respond_with_` methods.
:param matcher: the matcher object
"""
def __init__(self, matcher: RequestMatcher):
self.matcher = matcher
self.request_handler: Callable[[Request], Response] | None = None
self._hooks: list[Callable[[Request, Response], Response]] = []
def with_post_hook(self, hook: Callable[[Request, Response], Response]):
self._hooks.append(hook)
return self
def respond(self, request: Request) -> Response:
"""
Calls the request handler registered for this object.
If no response was specified previously, it raises
:py:class:`NoHandlerError` exception.
:param request: the incoming request object
:return: the response object
"""
if self.request_handler is None:
raise NoHandlerError(
"Matching request handler found but no response defined: {} {}".format(request.method, request.path)
)
else:
response = self.request_handler(request)
for hook in self._hooks:
response = hook(request, response)
return response
def respond_with_handler(self, func: Callable[[Request], Response]):
"""
Registers the specified function as a responder.
The function will receive the request object and must return with the response object.
"""
self.request_handler = func
def respond_with_response(self, response: Response):
self.request_handler = lambda request: response
def __repr__(self) -> str:
class_name = self.__class__.__name__
retval = (
f"<{class_name} uri={self.matcher.uri!r} method={self.matcher.method!r} "
f"query_string={self.matcher.query_string!r} headers={self.matcher.headers!r} data={self.matcher.data!r} "
f"json={self.matcher.json!r}>"
)
return retval
class RequestHandlerList(list[RequestHandler]):
"""
Represents a list of :py:class:`RequestHandler` objects.
"""
def match(self, request: Request) -> RequestHandler | None:
"""
Returns the first request handler which matches the specified request. Otherwise, it returns `None`.
"""
for requesthandler in self:
if requesthandler.matcher.match(request):
return requesthandler
return None
class HandlerType(Enum):
PERMANENT = "permanent"
ONESHOT = "oneshot"
ORDERED = "ordered"
class HTTPServerBase(abc.ABC): # pylint: disable=too-many-instance-attributes
"""
Abstract HTTP server with error handling.
:param host: the host or IP where the server will listen
:param port: the TCP port where the server will listen
:param ssl_context: the ssl context object to use for https connections
:param threaded: whether to handle concurrent requests in separate threads
.. py:attribute:: log
Attribute containing the list of two-element tuples. Each tuple contains
:py:class:`werkzeug.Request` and :py:class:`werkzeug.Response` object which represents the
incoming request and the outgoing response which happened during the lifetime
of the server.
.. py:attribute:: no_handler_status_code
Attribute containing the http status code (int) which will be the response
status when no matcher is found for the request. By default, it is set to *500*
but it can be overridden to any valid http status code such as *404* if needed.
"""
def __init__(
self,
host: str,
port: int,
ssl_context: SSLContext | None = None,
*,
threaded: bool = False,
):
"""
Initializes the instance.
"""
self.host = host
self.port = port
self.server: BaseWSGIServer | None = None
self.server_thread: threading.Thread | None = None
self.assertions: list[str | AssertionError] = []
self.handler_errors: list[Exception] = []
self.log: list[tuple[Request, Response]] = []
self.ssl_context = ssl_context
self.threaded = threaded
self.no_handler_status_code = 500
def __repr__(self):
return f"<{self.__class__.__name__} host={self.host} port={self.port}>"
def clear(self):
"""
Clears and resets the state attributes of the object.
This method is useful when the object needs to be re-used but stopping the server is not feasible.
"""
self.clear_assertions()
self.clear_handler_errors()
self.clear_log()
self.no_handler_status_code = 500
def clear_assertions(self):
"""
Clears the list of assertions
"""
self.assertions = []
def clear_handler_errors(self):
"""
Clears the list of collected errors from handler invocations
"""
self.handler_errors = []
def clear_log(self):
"""
Clears the list of log entries
"""
self.log = []
def url_for(self, suffix: str):
"""
Return an url for a given suffix.
This basically means that it prepends the string ``http://$HOST:$PORT/`` to the `suffix` parameter
(where $HOST and $PORT are the parameters given to the constructor).
When host is an IPv6 address, the required square brackets will be added
to it, forming a valid URL.
When SSL or TLS is in use, the protocol of the returned URL will be ``https``.
:param suffix: the suffix which will be added to the base url. It can start with ``/`` (slash) or
not, the url will be the same.
:return: the full url which refers to the server
"""
if not suffix.startswith("/"):
suffix = "/" + suffix
if self.ssl_context is None:
protocol = "http"
else:
protocol = "https"
host = self.format_host(self.host)
return "{}://{}:{}{}".format(protocol, host, self.port, suffix)
def create_matcher(self, *args, **kwargs) -> RequestMatcher:
"""
Creates a :py:class:`.RequestMatcher` instance with the specified parameters.
This method can be overridden if you want to use your own matcher.
"""
return RequestMatcher(*args, **kwargs)
def thread_target(self):
"""
This method serves as a thread target when the server is started.
This should not be called directly, but can be overridden to tailor it to your needs.
"""
assert self.server is not None
self.server.serve_forever()
def is_running(self) -> bool:
"""
Returns `True` when the server is running, otherwise `False`.
"""
return bool(self.server)
def start(self) -> None:
"""
Start the server in a thread.
This method returns immediately (e.g. does not block), and it's the caller's
responsibility to stop the server (by calling :py:meth:`stop`) when it is no longer needed).
If the server is not stopped by the caller and execution reaches the end, the
program needs to be terminated by Ctrl+C or by signal as it will not terminate until
the thread is stopped.
If the server is already running :py:class:`HTTPServerError` will be raised. If you are
unsure, call :py:meth:`is_running` first.
There's a context interface of this class which stops the server when the context block ends.
"""
if self.is_running():
raise HTTPServerError("Server is already running")
app = Request.application(self.application)
self.server = make_server(
self.host,
self.port,
app,
ssl_context=self.ssl_context,
threaded=self.threaded,
)
self.port = self.server.port # Update port (needed if `port` was set to 0)
# Explicitly make the new thread daemonic to avoid shutdown issues
self.server_thread = threading.Thread(target=self.thread_target, daemon=True)
self.server_thread.start()
def stop(self):
"""
Stop the running server.
Notifies the server thread about the intention of the stopping, and the thread will
terminate itself. This needs about 0.5 seconds in worst case.
Only a running server can be stopped. If the sever is not running, :py:class`HTTPServerError`
will be raised.
"""
assert self.server is not None
assert self.server_thread is not None
if not self.is_running():
raise HTTPServerError("Server is not running")
self.server.shutdown()
self.server_thread.join()
self.server = None
self.server_thread = None
def add_assertion(self, obj: str | AssertionError):
"""
Add a new assertion
Assertions can be added here, and when :py:meth:`check_assertions` is called,
it will raise AssertionError for pytest with the object specified here.
:param obj: An AssertionError, or an object which will be passed to an AssertionError.
"""
self.assertions.append(obj)
def check(self):
"""
Raises AssertionError or Errors raised in handlers.
Runs both :py:meth:`check_assertions` and :py:meth:`check_handler_errors`
"""
self.check_assertions()
self.check_handler_errors()
def check_assertions(self):
"""
Raise AssertionError when at least one assertion added
The first assertion added by :py:meth:`add_assertion` will be raised and
it will be removed from the list.
This method can be useful to get some insights into the errors happened in
the sever, and to have a proper error reporting in pytest.
"""
if self.assertions:
assertion = self.assertions.pop(0)
if isinstance(assertion, AssertionError):
raise assertion
raise AssertionError(assertion)
def check_handler_errors(self):
"""
Re-Raises any errors caused in request handlers
The first error raised by a handler will be re-raised here, and then
removed from the list.
"""
if self.handler_errors:
raise self.handler_errors.pop(0)
def respond_nohandler(self, request: Request, extra_message: str = ""):
"""
Add a 'no handler' assertion.
This method is called when the server wasn't able to find any handler to serve the request.
As the result, there's an assertion added (which can be raised by :py:meth:`check_assertions`).
"""
text = "No handler found for request {!r} with data {!r}.".format(request, request.data)
self.add_assertion(text + extra_message)
return Response(text + extra_message, self.no_handler_status_code)
@abc.abstractmethod
def dispatch(self, request: Request) -> Response:
"""
Dispatch a request to the appropriate request handler.
:param request: the request object from the werkzeug library
:return: the response object what the handler responded, or a response which contains the error
"""
def application(self, request: Request) -> Response:
"""
Entry point of werkzeug.
This method is called for each request, and it then calls the undecorated
:py:meth:`dispatch` method to serve the request.
:param request: the request object from the werkzeug library
:return: the response object what the dispatch returned
"""
request.get_data()
response = self.dispatch(request)
self.log.append((request, response))
return response
def __enter__(self):
"""
Provide the context API
It starts the server in a thread if the server is not already running.
"""
if not self.is_running():
self.start()
return self
def __exit__(
self,
exc_type: type[BaseException] | None,
exc_value: BaseException | None,
traceback: TracebackType | None,
):
"""
Provide the context API
It stops the server if the server is running.
Please note that depending on the internal things of werkzeug, it may take 0.5 seconds.
"""
if self.is_running():
self.stop()
@staticmethod
def format_host(host: str):
"""
Formats a hostname so it can be used in a URL.
Notably, this adds brackets around IPV6 addresses when
they are missing.
"""
try:
ipaddress.IPv6Address(host)
is_ipv6 = True
except ValueError:
is_ipv6 = False
if is_ipv6 and not host.startswith("[") and not host.endswith("]"):
return f"[{host}]"
return host
class HTTPServer(HTTPServerBase): # pylint: disable=too-many-instance-attributes
"""
Server instance which manages handlers to serve pre-defined requests.
:param host: the host or IP where the server will listen
:param port: the TCP port where the server will listen
:param ssl_context: the ssl context object to use for https connections
:param default_waiting_settings: the waiting settings object to use as default settings for :py:meth:`wait` context
manager
:param threaded: whether to handle concurrent requests in separate threads
.. py:attribute:: no_handler_status_code
Attribute containing the http status code (int) which will be the response
status when no matcher is found for the request. By default, it is set to *500*
but it can be overridden to any valid http status code such as *404* if needed.
"""
DEFAULT_LISTEN_HOST = "localhost"
DEFAULT_LISTEN_PORT = 0 # Use ephemeral port
def __init__(
self,
host: str = DEFAULT_LISTEN_HOST,
port: int = DEFAULT_LISTEN_PORT,
ssl_context: SSLContext | None = None,
default_waiting_settings: WaitingSettings | None = None,
*,
threaded: bool = False,
):
"""
Initializes the instance.
"""
super().__init__(host, port, ssl_context, threaded=threaded)
self.ordered_handlers: list[RequestHandler] = []
self.oneshot_handlers = RequestHandlerList()
self.handlers = RequestHandlerList()
self.permanently_failed = False
if default_waiting_settings is not None:
self.default_waiting_settings = default_waiting_settings
else:
self.default_waiting_settings = WaitingSettings()
self._waiting_settings = copy(self.default_waiting_settings)
self._waiting_result: queue.LifoQueue[bool] = queue.LifoQueue(maxsize=1)
def clear(self):
"""
Clears and resets the state attributes of the object.
This method is useful when the object needs to be re-used but stopping the server is not feasible.
"""
super().clear()
self.clear_all_handlers()
self.permanently_failed = False
def clear_all_handlers(self):
"""
Clears all types of the handlers (ordered, oneshot, permanent)
"""
self.ordered_handlers = []
self.oneshot_handlers = RequestHandlerList()
self.handlers = RequestHandlerList()
def expect(self, matcher: RequestMatcher, handler_type: HandlerType = HandlerType.PERMANENT) -> RequestHandler:
"""
Create and register a request handler.
:param matcher: :py:class:`RequestMatcher` used to match requests.
:param handler_type: type of handler
"""
request_handler = RequestHandler(matcher)
if handler_type == HandlerType.PERMANENT:
self.handlers.append(request_handler)
elif handler_type == HandlerType.ONESHOT:
self.oneshot_handlers.append(request_handler)
elif handler_type == HandlerType.ORDERED:
self.ordered_handlers.append(request_handler)
return request_handler
def expect_request(
self,
uri: str | URIPattern | Pattern[str],
method: str = METHOD_ALL,
data: str | bytes | None = None,
data_encoding: str = "utf-8",
headers: Mapping[str, str] | None = None,
query_string: None | QueryMatcher | str | bytes | Mapping[str, str] = None,
header_value_matcher: HVMATCHER_T | None = None,
handler_type: HandlerType = HandlerType.PERMANENT,
json: Any = UNDEFINED,
) -> RequestHandler:
"""
Create and register a request handler.
If `handler_type` is `HandlerType.PERMANENT` a permanent request handler is created. This handler can be used as
many times as the request matches it, but ordered handlers have higher priority so if there's one or more
ordered handler registered, those must be used first.
If `handler_type` is `HandlerType.ONESHOT` a oneshot request handler is created. This handler can be only used
once. Once the server serves a response for this handler, the handler will be dropped.
If `handler_type` is `HandlerType.ORDERED` an ordered request handler is created. Comparing to oneshot handler,
ordered handler also determines the order of the requests to be served. For example if there are two ordered
handlers registered, the first request must hit the first handler, and the second request must hit the second
one, and not vice versa. If one or more ordered handler defined, those
must be exhausted first.
.. note::
Once this method is called, the response should also be specified by
calling one of the respond methods of the returned
:py:class:`RequestHandler` object, otherwise
:py:class:`NoHandlerError` will be raised on an incoming request.
:param uri: URI of the request. This must be an absolute path starting with ``/``, a
:py:class:`URIPattern` object, or a regular expression compiled by :py:func:`re.compile`.
:param method: HTTP method of the request. If not specified (or `METHOD_ALL`
specified), all HTTP requests will match. Case insensitive.
:param data: payload of the HTTP request. This could be a string (utf-8 encoded
by default, see `data_encoding`) or a bytes object.
:param data_encoding: the encoding used for data parameter if data is a string.
:param headers: dictionary of the headers of the request to be matched
:param query_string: the http query string, after ``?``, such as ``username=user``.
If string is specified it will be encoded to bytes with the encode method of
the string. If dict is specified, it will be matched to the ``key=value`` pairs
specified in the request. If multiple values specified for a given key, the first
value will be used. If multiple values needed to be handled, use ``MultiDict``
object from werkzeug.
:param header_value_matcher: :py:class:`HeaderValueMatcher` that matches
values of headers, or a ``Callable[[str, Optional[str], str], bool]``
receiving the header key (from `headers`), header value (or `None`) and the expected
value (from `headers`) and should return ``True`` if the header matches, ``False`` otherwise.
:param handler_type: type of handler
:param json: a python object (eg. a dict) whose value will be compared to the request body after it
is loaded as json. If load fails, this matcher will be failed also. *Content-Type* is not checked.
If that's desired, add it to the headers parameter.
:return: Created and register :py:class:`RequestHandler`.
Parameters `json` and `data` are mutually exclusive.
"""
matcher = self.create_matcher(
uri,
method=method.upper(),
data=data,
data_encoding=data_encoding,
headers=headers,
query_string=query_string,
header_value_matcher=header_value_matcher,
json=json,
)
request_handler = RequestHandler(matcher)
if handler_type == HandlerType.PERMANENT:
self.handlers.append(request_handler)
elif handler_type == HandlerType.ONESHOT:
self.oneshot_handlers.append(request_handler)
elif handler_type == HandlerType.ORDERED:
self.ordered_handlers.append(request_handler)
return request_handler
def expect_oneshot_request(
self,
uri: str | URIPattern | Pattern[str],
method: str = METHOD_ALL,
data: str | bytes | None = None,
data_encoding: str = "utf-8",
headers: Mapping[str, str] | None = None,
query_string: None | QueryMatcher | str | bytes | Mapping[str, str] = None,
header_value_matcher: HVMATCHER_T | None = None,
json: Any = UNDEFINED,
) -> RequestHandler:
"""
Create and register a oneshot request handler.
This is a method for convenience. See :py:meth:`expect_request` for documentation.
:param uri: URI of the request. This must be an absolute path starting with ``/``, a
:py:class:`URIPattern` object, or a regular expression compiled by :py:func:`re.compile`.
:param method: HTTP method of the request. If not specified (or `METHOD_ALL`
specified), all HTTP requests will match.
:param data: payload of the HTTP request. This could be a string (utf-8 encoded
by default, see `data_encoding`) or a bytes object.
:param data_encoding: the encoding used for data parameter if data is a string.
:param headers: dictionary of the headers of the request to be matched
:param query_string: the http query string, after ``?``, such as ``username=user``.
If string is specified it will be encoded to bytes with the encode method of
the string. If dict is specified, it will be matched to the ``key=value`` pairs
specified in the request. If multiple values specified for a given key, the first
value will be used. If multiple values needed to be handled, use ``MultiDict``
object from werkzeug.
:param header_value_matcher: :py:class:`HeaderValueMatcher` that matches
values of headers, or a ``Callable[[str, Optional[str], str], bool]``
receiving the header key (from `headers`), header value (or `None`) and the expected
value (from `headers`) and should return ``True`` if the header matches, ``False`` otherwise.
:param json: a python object (eg. a dict) whose value will be compared to the request body after it
is loaded as json. If load fails, this matcher will be failed also. *Content-Type* is not checked.
If that's desired, add it to the headers parameter.
:return: Created and register :py:class:`RequestHandler`.
Parameters `json` and `data` are mutually exclusive.
"""
return self.expect_request(
uri=uri,
method=method,
data=data,
data_encoding=data_encoding,
headers=headers,
query_string=query_string,
header_value_matcher=header_value_matcher,
handler_type=HandlerType.ONESHOT,
json=json,
)
def expect_ordered_request(
self,
uri: str | URIPattern | Pattern[str],
method: str = METHOD_ALL,
data: str | bytes | None = None,
data_encoding: str = "utf-8",
headers: Mapping[str, str] | None = None,
query_string: None | QueryMatcher | str | bytes | Mapping[str, str] = None,
header_value_matcher: HVMATCHER_T | None = None,
json: Any = UNDEFINED,
) -> RequestHandler:
"""
Create and register a ordered request handler.
This is a method for convenience. See :py:meth:`expect_request` for documentation.
:param uri: URI of the request. This must be an absolute path starting with ``/``, a
:py:class:`URIPattern` object, or a regular expression compiled by :py:func:`re.compile`.
:param method: HTTP method of the request. If not specified (or `METHOD_ALL`
specified), all HTTP requests will match.
:param data: payload of the HTTP request. This could be a string (utf-8 encoded
by default, see `data_encoding`) or a bytes object.
:param data_encoding: the encoding used for data parameter if data is a string.
:param headers: dictionary of the headers of the request to be matched
:param query_string: the http query string, after ``?``, such as ``username=user``.
If string is specified it will be encoded to bytes with the encode method of
the string. If dict is specified, it will be matched to the ``key=value`` pairs
specified in the request. If multiple values specified for a given key, the first
value will be used. If multiple values needed to be handled, use ``MultiDict``
object from werkzeug.
:param header_value_matcher: :py:class:`HeaderValueMatcher` that matches
values of headers, or a ``Callable[[str, Optional[str], str], bool]``
receiving the header key (from `headers`), header value (or `None`) and the expected
value (from `headers`) and should return ``True`` if the header matches, ``False`` otherwise.
:param json: a python object (eg. a dict) whose value will be compared to the request body after it
is loaded as json. If load fails, this matcher will be failed also. *Content-Type* is not checked.
If that's desired, add it to the headers parameter.
:return: Created and register :py:class:`RequestHandler`.
Parameters `json` and `data` are mutually exclusive.
"""
return self.expect_request(
uri=uri,
method=method,
data=data,
data_encoding=data_encoding,
headers=headers,
query_string=query_string,
header_value_matcher=header_value_matcher,
handler_type=HandlerType.ORDERED,
json=json,
)
def format_matchers(self) -> str:
"""
Return a string representation of the matchers
This method returns a human-readable string representation of the matchers
registered. You can observe which requests will be served, etc.
This method is primarily used when reporting errors.
"""
def format_handlers(handlers: list[RequestHandler]):
if handlers:
return [" {!r}".format(handler.matcher) for handler in handlers]
else:
return [" none"]
lines: list[str] = []
lines.append("Ordered matchers:")
lines.extend(format_handlers(self.ordered_handlers))
lines.append("")
lines.append("Oneshot matchers:")
lines.extend(format_handlers(self.oneshot_handlers))
lines.append("")
lines.append("Persistent matchers:")
lines.extend(format_handlers(self.handlers))
return "\n".join(lines)
def respond_nohandler(self, request: Request, extra_message: str = ""):
"""
Add a 'no handler' assertion.
This method is called when the server wasn't able to find any handler to serve the request.
As the result, there's an assertion added (which can be raised by :py:meth:`check_assertions`).
"""
if self._waiting_settings.stop_on_nohandler:
self._set_waiting_result(value=False)
return super().respond_nohandler(request, self.format_matchers() + extra_message)
def respond_permanent_failure(self):
"""
Add a 'permanent failure' assertion.
This assertion means that no further requests will be handled. This is the resuld of missing
an ordered matcher.
"""
self.add_assertion("All requests will be permanently failed due failed ordered handler")
return Response("No handler found for this request", 500)
def dispatch(self, request: Request) -> Response:
"""
Dispatch a request to the appropriate request handler.
This method tries to find the request handler whose matcher matches the request, and
then calls it in order to serve the request.
First, the request is checked for the ordered matchers. If there's an ordered matcher,
it must match the request, otherwise the server will be put into a `permanent failure`
mode in which it makes all request failed - this is the intended way of working of ordered
matchers.
Then oneshot handlers, and the permanent handlers are looked up.
:param request: the request object from the werkzeug library
:return: the response object what the handler responded, or a response which contains the error
"""
if self.permanently_failed:
return self.respond_permanent_failure()
handler = None
if self.ordered_handlers:
handler = self.ordered_handlers[0]
if not handler.matcher.match(request):
self.permanently_failed = True
response = self.respond_nohandler(request)
return response
self.ordered_handlers.pop(0)
self._update_waiting_result()
if not handler:
handler = self.oneshot_handlers.match(request)
if handler:
self.oneshot_handlers.remove(handler)
self._update_waiting_result()
else:
handler = self.handlers.match(request)
if not handler:
return self.respond_nohandler(request)
try:
response = handler.respond(request)
except Error:
# don't collect package-internal errors
raise
except AssertionError as e:
self.add_assertion(e)
raise
except Exception as e:
self.handler_errors.append(e)
raise
if response is None:
response = Response("")
if isinstance(response, str):
response = Response(response)
return response
def _set_waiting_result(self, value: bool) -> None: # noqa: FBT001
"""Set waiting_result
Setting is implemented as putting value to queue without waiting. If queue is full we simply ignore the
exception, because that means that waiting_result was already set, but not read.
"""
with suppress(queue.Full):
self._waiting_result.put_nowait(value)
def _update_waiting_result(self) -> None:
if not self.oneshot_handlers and not self.ordered_handlers:
self._set_waiting_result(value=True)
@contextmanager
def wait(
self,
raise_assertions: bool | None = None,
stop_on_nohandler: bool | None = None,
timeout: float | None = None,
):
"""Context manager to wait until the first of following event occurs: all ordered and oneshot handlers were
executed, unexpected request was received (if `stop_on_nohandler` is set to `True`), or time was out
:param raise_assertions: whether raise assertions on unexpected request or timeout or not
:param stop_on_nohandler: whether stop on unexpected request or not
:param timeout: time (in seconds) until time is out
Example:
.. code-block:: python
def test_wait(httpserver):
httpserver.expect_oneshot_request("/").respond_with_data("OK")
with httpserver.wait(
raise_assertions=False, stop_on_nohandler=False, timeout=1
) as waiting:
requests.get(httpserver.url_for("/"))
# `waiting` is :py:class:`Waiting`
assert waiting.result
print("Elapsed time: {} sec".format(waiting.elapsed_time))
"""
if raise_assertions is None:
self._waiting_settings.raise_assertions = self.default_waiting_settings.raise_assertions
else:
self._waiting_settings.raise_assertions = raise_assertions
if stop_on_nohandler is None:
self._waiting_settings.stop_on_nohandler = self.default_waiting_settings.stop_on_nohandler
else:
self._waiting_settings.stop_on_nohandler = stop_on_nohandler
if timeout is None:
self._waiting_settings.timeout = self.default_waiting_settings.timeout
else:
self._waiting_settings.timeout = timeout
# Ensure that waiting_result is empty
with suppress(queue.Empty):
self._waiting_result.get_nowait()
waiting = Waiting()
yield waiting
try:
waiting_result = self._waiting_result.get(timeout=self._waiting_settings.timeout)
waiting.complete(result=waiting_result)
except queue.Empty:
waiting.complete(result=False)
if self._waiting_settings.raise_assertions:
raise AssertionError(
"Wait timeout occurred, but some handlers left:\n{}".format(self.format_matchers())
)
if self._waiting_settings.raise_assertions and not waiting.result:
self.check_assertions()
def iter_matching_requests(self, matcher: RequestMatcher) -> Iterable[tuple[Request, Response]]:
"""
Queries log for matching requests.
:param matcher: the matcher object to match requests
:return: an iterator with request-response pair from the log
"""
for request, response in self.log:
if matcher.match(request):
yield (request, response)
def get_matching_requests_count(self, matcher: RequestMatcher) -> int:
"""
Queries the log for matching requests, returning the number of log
entries matching for the specified matcher.
:param matcher: the matcher object to match requests
:return: the number of log entries matching
"""
return len(list(self.iter_matching_requests(matcher)))
def assert_request_made(self, matcher: RequestMatcher, *, count: int = 1):
"""
Check the amount of log entries matching for the matcher specified. By
default it verifies that exactly one request matching for the matcher
specified. The expected count can be customized with the count kwarg
(including zero, which asserts that no requests made for the given
matcher).
:param matcher: the matcher object to match requests
:param count: the expected number of matches in the log
:return: ``None`` if the assert succeeded, raises
:py:class:`AssertionError` if not.
"""
matching_count = self.get_matching_requests_count(matcher)
if matching_count != count:
similar_requests: list[Request] = []
for request, _ in self.log:
if request.path == matcher.uri:
similar_requests.append(request)
assert_msg_lines = [
f"Matching request found {matching_count} times but expected {count} times.",
f"Expected request: {matcher}",
]
if similar_requests:
assert_msg_lines.append(f"Found {len(similar_requests)} similar request(s):")
for request in similar_requests:
assert_msg_lines.extend(
(
"--- Similar Request Start",
f"Path: {request.path}",
f"Method: {request.method}",
f"Body: {request.get_data()!r}",
f"Headers: {request.headers}",
f"Query String: {request.query_string.decode('utf-8')!r}",
"--- Similar Request End",
)
)
else:
assert_msg_lines.append("No similar requests found.")
assert_msg = "\n".join(assert_msg_lines) + "\n"
assert matching_count == count, assert_msg
|