1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
|
Metadata-Version: 2.1
Name: graypy
Version: 2.1.0
Summary: Python logging handlers that send messages in the Graylog Extended Log Format (GELF).
Home-page: https://github.com/severb/graypy
Author: Sever Banesiu
Author-email: banesiu.sever@gmail.com
License: BSD License
Description: ######
graypy
######
.. image:: https://img.shields.io/pypi/v/graypy.svg
:target: https://pypi.python.org/pypi/graypy
:alt: PyPI Status
.. image:: https://travis-ci.org/severb/graypy.svg?branch=master
:target: https://travis-ci.org/severb/graypy
:alt: Build Status
.. image:: https://readthedocs.org/projects/graypy/badge/?version=stable
:target: https://graypy.readthedocs.io/en/stable/?badge=stable
:alt: Documentation Status
.. image:: https://codecov.io/gh/severb/graypy/branch/master/graph/badge.svg
:target: https://codecov.io/gh/severb/graypy
:alt: Coverage Status
Description
===========
Python logging handlers that send log messages in the
Graylog Extended Log Format (GELF_).
graypy supports sending GELF logs to both Graylog2 and Graylog3 servers.
Installing
==========
Using pip
---------
Install the basic graypy python logging handlers:
.. code-block:: console
pip install graypy
Install with requirements for ``GELFRabbitHandler``:
.. code-block:: console
pip install graypy[amqp]
Using easy_install
------------------
Install the basic graypy python logging handlers:
.. code-block:: console
easy_install graypy
Install with requirements for ``GELFRabbitHandler``:
.. code-block:: console
easy_install graypy[amqp]
Usage
=====
graypy sends GELF logs to a Graylog server via subclasses of the python
`logging.Handler`_ class.
Below is the list of ready to run GELF logging handlers defined by graypy:
* ``GELFUDPHandler`` - UDP log forwarding
* ``GELFTCPHandler`` - TCP log forwarding
* ``GELFTLSHandler`` - TCP log forwarding with TLS support
* ``GELFHTTPHandler`` - HTTP log forwarding
* ``GELFRabbitHandler`` - RabbitMQ log forwarding
UDP Logging
-----------
UDP Log forwarding to a locally hosted Graylog server can be easily done with
the ``GELFUDPHandler``:
.. code-block:: python
import logging
import graypy
my_logger = logging.getLogger('test_logger')
my_logger.setLevel(logging.DEBUG)
handler = graypy.GELFUDPHandler('localhost', 12201)
my_logger.addHandler(handler)
my_logger.debug('Hello Graylog.')
UDP GELF Chunkers
^^^^^^^^^^^^^^^^^
`GELF UDP Chunking`_ is supported by the ``GELFUDPHandler`` and is defined by
the ``gelf_chunker`` argument within its constructor. By default the
``GELFWarningChunker`` is used, thus, GELF messages that chunk overflow
(i.e. consisting of more than 128 chunks) will issue a
``GELFChunkOverflowWarning`` and **will be dropped**.
Other ``gelf_chunker`` options are also available:
* ``BaseGELFChunker`` silently drops GELF messages that chunk overflow
* ``GELFTruncatingChunker`` issues a ``GELFChunkOverflowWarning`` and
simplifies and truncates GELF messages that chunk overflow in a attempt
to send some content to Graylog. If this process fails to prevent
another chunk overflow a ``GELFTruncationFailureWarning`` is issued.
RabbitMQ Logging
----------------
Alternately, use ``GELFRabbitHandler`` to send messages to RabbitMQ and
configure your Graylog server to consume messages via AMQP. This prevents log
messages from being lost due to dropped UDP packets (``GELFUDPHandler`` sends
messages to Graylog using UDP). You will need to configure RabbitMQ with a
``gelf_log`` queue and bind it to the ``logging.gelf`` exchange so messages
are properly routed to a queue that can be consumed by Graylog (the queue and
exchange names may be customized to your liking).
.. code-block:: python
import logging
import graypy
my_logger = logging.getLogger('test_logger')
my_logger.setLevel(logging.DEBUG)
handler = graypy.GELFRabbitHandler('amqp://guest:guest@localhost/', exchange='logging.gelf')
my_logger.addHandler(handler)
my_logger.debug('Hello Graylog.')
Django Logging
--------------
It's easy to integrate ``graypy`` with Django's logging settings. Just add a
new handler in your ``settings.py``:
.. code-block:: python
LOGGING = {
'version': 1,
# other dictConfig keys here...
'handlers': {
'graypy': {
'level': 'WARNING',
'class': 'graypy.GELFUDPHandler',
'host': 'localhost',
'port': 12201,
},
},
'loggers': {
'django.request': {
'handlers': ['graypy'],
'level': 'ERROR',
'propagate': True,
},
},
}
Traceback Logging
-----------------
By default log captured exception tracebacks are added to the GELF log as
``full_message`` fields:
.. code-block:: python
import logging
import graypy
my_logger = logging.getLogger('test_logger')
my_logger.setLevel(logging.DEBUG)
handler = graypy.GELFUDPHandler('localhost', 12201)
my_logger.addHandler(handler)
try:
puff_the_magic_dragon()
except NameError:
my_logger.debug('No dragons here.', exc_info=1)
Default Logging Fields
----------------------
By default a number of debugging logging fields are automatically added to the
GELF log if available:
* function
* pid
* process_name
* thread_name
You can disable automatically adding these debugging logging fields by
specifying ``debugging_fields=False`` in the handler's constructor:
.. code-block:: python
handler = graypy.GELFUDPHandler('localhost', 12201, debugging_fields=False)
Adding Custom Logging Fields
----------------------------
graypy also supports including custom fields in the GELF logs sent to Graylog.
This can be done by using Python's LoggerAdapter_ and Filter_ classes.
Using LoggerAdapter
^^^^^^^^^^^^^^^^^^^
LoggerAdapter_ makes it easy to add static information to your GELF log
messages:
.. code-block:: python
import logging
import graypy
my_logger = logging.getLogger('test_logger')
my_logger.setLevel(logging.DEBUG)
handler = graypy.GELFUDPHandler('localhost', 12201)
my_logger.addHandler(handler)
my_adapter = logging.LoggerAdapter(logging.getLogger('test_logger'),
{'username': 'John'})
my_adapter.debug('Hello Graylog from John.')
Using Filter
^^^^^^^^^^^^
Filter_ gives more flexibility and allows for dynamic information to be
added to your GELF logs:
.. code-block:: python
import logging
import graypy
class UsernameFilter(logging.Filter):
def __init__(self):
# In an actual use case would dynamically get this
# (e.g. from memcache)
self.username = 'John'
def filter(self, record):
record.username = self.username
return True
my_logger = logging.getLogger('test_logger')
my_logger.setLevel(logging.DEBUG)
handler = graypy.GELFUDPHandler('localhost', 12201)
my_logger.addHandler(handler)
my_logger.addFilter(UsernameFilter())
my_logger.debug('Hello Graylog from John.')
Contributors
============
* Sever Banesiu
* Daniel Miller
* Tushar Makkar
* Nathan Klapstein
.. _GELF: https://docs.graylog.org/en/latest/pages/gelf.html
.. _logging.Handler: https://docs.python.org/3/library/logging.html#logging.Handler
.. _GELF UDP Chunking: https://docs.graylog.org/en/latest/pages/gelf.html#chunking
.. _LoggerAdapter: https://docs.python.org/howto/logging-cookbook.html#using-loggeradapters-to-impart-contextual-information
.. _Filter: https://docs.python.org/howto/logging-cookbook.html#using-filters-to-impart-contextual-information
Keywords: logging gelf graylog2 graylog udp amqp
Platform: UNKNOWN
Classifier: License :: OSI Approved :: BSD License
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.2
Classifier: Programming Language :: Python :: 3.3
Classifier: Programming Language :: Python :: 3.4
Classifier: Programming Language :: Python :: 3.5
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: Implementation :: PyPy
Classifier: Topic :: System :: Logging
Description-Content-Type: text/x-rst
Provides-Extra: docs
Provides-Extra: amqp
|
