1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190
|
=====================
JSON log formatter 🪵
=====================
The library helps you to store logs in JSON format. Why is it important?
Well, it facilitates integration with **Logstash**.
Usage example:
.. code-block:: python
import logging
import json_log_formatter
formatter = json_log_formatter.JSONFormatter()
json_handler = logging.FileHandler(filename='/var/log/my-log.json')
json_handler.setFormatter(formatter)
logger = logging.getLogger('my_json')
logger.addHandler(json_handler)
logger.setLevel(logging.INFO)
logger.info('Sign up', extra={'referral_code': '52d6ce'})
try:
raise ValueError('something wrong')
except ValueError:
logger.error('Request failed', exc_info=True)
The log file will contain the following log record (inline).
.. code-block:: json
{
"message": "Sign up",
"time": "2015-09-01T06:06:26.524448",
"referral_code": "52d6ce"
}
{
"message": "Request failed",
"time": "2015-09-01T06:06:26.524449",
"exc_info": "Traceback (most recent call last): ..."
}
If you use a log collection and analysis system,
you might need to include the built-in
`log record attributes <https://docs.python.org/3/library/logging.html#logrecord-attributes>`_
with ``VerboseJSONFormatter``.
.. code-block:: python
json_handler.setFormatter(json_log_formatter.VerboseJSONFormatter())
logger.error('An error has occured')
.. code-block:: json
{
"filename": "tests.py",
"funcName": "test_file_name_is_testspy",
"levelname": "ERROR",
"lineno": 276,
"module": "tests",
"name": "my_json",
"pathname": "/Users/bob/json-log-formatter/tests.py",
"process": 3081,
"processName": "MainProcess",
"stack_info": null,
"thread": 4664270272,
"threadName": "MainThread",
"message": "An error has occured",
"time": "2021-07-04T21:05:42.767726"
}
If you need to flatten complex objects as strings, use ``FlatJSONFormatter``.
.. code-block:: python
json_handler.setFormatter(json_log_formatter.FlatJSONFormatter())
logger.error('An error has occured')
logger.info('Sign up', extra={'request': WSGIRequest({
'PATH_INFO': 'bogus',
'REQUEST_METHOD': 'bogus',
'CONTENT_TYPE': 'text/html; charset=utf8',
'wsgi.input': BytesIO(b''),
})})
.. code-block:: json
{
"message": "Sign up",
"time": "2024-10-01T00:59:29.332888+00:00",
"request": "<WSGIRequest: BOGUS '/bogus'>"
}
JSON libraries
--------------
You can use **ujson** or **simplejson** instead of built-in **json** library.
.. code-block:: python
import json_log_formatter
import ujson
formatter = json_log_formatter.JSONFormatter()
formatter.json_lib = ujson
Note, **ujson** doesn't support ``dumps(default=f)`` argument:
if it can't serialize an attribute, it might fail with ``TypeError`` or skip an attribute.
Django integration
------------------
Here is an example of how the JSON formatter can be used with Django.
.. code-block:: python
LOGGING['formatters']['json'] = {
'()': 'json_log_formatter.JSONFormatter',
}
LOGGING['handlers']['json_file'] = {
'level': 'INFO',
'class': 'logging.FileHandler',
'filename': '/var/log/my-log.json',
'formatter': 'json',
}
LOGGING['loggers']['my_json'] = {
'handlers': ['json_file'],
'level': 'INFO',
}
Let's try to log something.
.. code-block:: python
import logging
logger = logging.getLogger('my_json')
logger.info('Sign up', extra={'referral_code': '52d6ce'})
Custom formatter
----------------
You will likely need a custom log formatter. For instance, you want to log
a user ID, an IP address and ``time`` as ``django.utils.timezone.now()``.
To do so you should override ``JSONFormatter.json_record()``.
.. code-block:: python
class CustomisedJSONFormatter(json_log_formatter.JSONFormatter):
def json_record(self, message: str, extra: dict, record: logging.LogRecord) -> dict:
extra['message'] = message
extra['user_id'] = current_user_id()
extra['ip'] = current_ip()
# Include builtins
extra['level'] = record.levelname
extra['name'] = record.name
if 'time' not in extra:
extra['time'] = django.utils.timezone.now()
if record.exc_info:
extra['exc_info'] = self.formatException(record.exc_info)
return extra
Let's say you want ``datetime`` to be serialized as timestamp.
You can use **ujson** (which does it by default) and disable
ISO8601 date mutation.
.. code-block:: python
class CustomisedJSONFormatter(json_log_formatter.JSONFormatter):
json_lib = ujson
def mutate_json_record(self, json_record):
return json_record
Tests
-----
.. code-block:: console
$ pip install -r requirements.txt
$ tox
|