1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908
|
*********
Templates
*********
Description
===========
Templates are a key feature of rsyslog. They allow to specify any format
a user might want. They are also used for dynamic file name generation.
Every output in rsyslog uses templates - this holds true for files, user
messages and so on. The database writer expects its template to be a
proper SQL statement - so this is highly customizable too. You might ask
how does all of this work when no templates at all are specified. Good
question ;). The answer is simple, though. Templates compatible with the
stock syslogd formats are hardcoded into rsyslogd. So if no template is
specified, we use one of those hardcoded templates. Search for
"template\_" in rsconf.c and you will find the hardcoded ones.
Templates are specified by template() objects. They can also be
specified via $template legacy statements.
.. note::
Note: key elements of templates are rsyslog properties. See the
:doc:`rsyslog properties reference <properties>` for a list of which
are available.
Template processing
===================
Due to lack of standardization regarding logs formats, when a template is
specified it's supposed to include HEADER, as defined in `RFC5424 <https://tools.ietf.org/html/rfc5424>`_
It's very important to have this in mind, and also to understand how
`rsyslog parsing <http://www.rsyslog.com/doc/syslog_parsing.html>`_ works.
For example, if the MSG field is set to "this:is a message" and neither HOSTNAME
nor TAG are specified, the outgoing parser will split the message as:
.. code-block:: none
TAG:this:
MSG:is a message
The template() object
=====================
The template() object is used to define templates. Note that it is a
**static** object, that means all templates are defined when rsyslog
reads the config file. As such, templates are not affected by
if-statements or config nesting.
The basic structure of the template object is as follows:
.. code-block:: none
template(parameters)
In addition to this simpler syntax, list templates (to be described
below) support an extended syntax:
.. code-block:: none
template(parameters) { list-descriptions }
Each template has a parameter **name**, which specifies the template
name, and a parameter **type**, which specifies the template type. The
name parameter must be unique, and behaviour is unpredictable if it is
not. The **type** parameter specifies different template types.
Different types simply enable different ways to specify the template
content. The template type **does not** affect what an (output) plugin
can do with it. So use the type that best fits your needs (from a config
writing point of view!). The following types are available:
- list
- subtree
- string
- plugin
The various types are described below.
List
----
In this case, the template is generated by a list of constant and
variable statements. These follow the template spec in curly braces.
This type is also primarily meant to be used with structure-aware outputs,
like ommongodb. However, it also works perfectly with text-based
outputs. We recommend to use this mode if more complex property
substitutions need to be done. In that case, the list-based template
syntax is much clearer than the simple string-based one.
The list template contains the template header (with **type="list"**)
and is followed by **constant** and **property** statements, given in
curly braces to signify the template statement they belong to. As the
name says, **constant** statements describe constant text and
**property** describes property access. There are many options to
**property**, described further below. Most of these options are used to
extract only partial property contents or to modify the text obtained
(for instance to change its case to upper or lower case).
To grasp the idea, an actual sample is:
.. code-block:: none
template(name="tpl1" type="list") {
constant(value="Syslog MSG is: '")
property(name="msg")
constant(value="', ")
property(name="timereported" dateFormat="rfc3339" caseConversion="lower")
constant(value="\n")
}
This sample is probably primarily targeted at the usual file-based
output.
Constant statement
^^^^^^^^^^^^^^^^^^
This provides a way to specify constant text. The text is used
literally. It is primarily intended for text-based output, so that some
constant text can be included. For example, if a complex template is
built for file output, one usually needs to finish it by a newline,
which can be introduced by a constant statement. Here is an actual
sample of that use case from the rsylsog testbench:
::
template(name="outfmt" type="list") {
property(name="$!usr!msgnum")
constant(value="\n")
}
The following escape sequences are recognized inside the constant text:
- \\\\ - single backslash
- \\n - LF
- \\ooo - (three octal digits) - represents a character with this
octal numerical value (e.g. \\101 equals "A"). Note that three octal digits
must be given (in contrast to some languages, where between one and
three are valid). While we support octal notation, we recommend to
use hex notation as this is better known.
- \\xhh - (where h is a hex digit) - represents a character with this
hexadecimal numerical value (e.g. \\x41 equals "A"). Note that two hexadecimal
digits must be given (in contrast to some languages where either one or two
are valid).
- ... some others ... list needs to be extended
Note: if an unsupported character follows a backslash, this is treated
as an error. Behaviour is unpredictable in this case.
To aid usage of the same template both for text-based outputs and
structured ones, constant text without an "outname" parameter will be
ignored when creating the name/value tree for structured outputs. So if
you want to supply some constant text e.g. to mongodb, you must include
an outname, as can be seen here:
.. code-block:: none
template(name="outfmt" type="list") {
property(name="$!usr!msgnum")
constant(value="\n" outname="IWantThisInMyDB")
}
To generate a constant json field, the `format` parameter can be used, as
in this example
.. code-block:: none
template(name="outfmt" type="list" option.jsonf="on") {
property(outname="message" name="msg" format="jsonf")
constant(outname="@version" value="1" format="jsonf")
}
The constant statement in this case will generate `"@version":"1"`. Note that to do
this, both the `value` and the `format` parameters must be given.
The "constant" statement supports the following parameters:
- value - the constant value to use
- outname - the output field name (for structured outputs)
- format - can be either empty or `jsonf`
Property statement
^^^^^^^^^^^^^^^^^^
This statement is used to include property values. It can access all
properties. Also, options permit to specify picking only part of a
property or modifying it. It supports the following parameters:
- **name** - the name of the property to access
- **outname** - the output field name (for structured outputs)
- **dateformat** - the date format to use (only for date-related properties).
`Here <property_replacer.html#property-options>`_ you can find a list of all
property options. **TODO:** right now, the property replacer documentation
contains property format options for string templates, only. The formats for
non-string templates differ. For example, date format options in string
templates start with "date-" whereas those in property statements do not
(e.g. "date-year" vs. just "year"). The technical reason behind this is
that inside string templates, the option must include what it applies
to whereas with the explicit format that is part of the parameter name.
To create a customised format you can use multiple property options
together. The following example would result in **YYYY-MM-DD**:
.. code-block:: none
property(name="timereported" dateformat="year")
constant(value="-")
property(name="timereported" dateformat="month")
constant(value="-")
property(name="timereported" dateformat="day")
- **date.inUTC** - date shall be shown in UTC (please note that this
requires a bit more performance due to the necessary conversions)
Available since 8.18.0.
- **caseconversion** - permits to convert case of the text. Supported
values are "lower" and "upper"
- **controlcharacters** - specifies how to handle control characters.
Supported values are "escape", which escapes them, "space", which
replaces them by a single space, and "drop", which simply removes
them from the string.
- **securepath** - used for creating pathnames suitable for use in dynafile
templates
- **format** - specify format on a field basis. Supported values are:
- "`csv <property_replacer.html#csv>`_\ " for use when csv-data is
generated
- "`json <property_replacer.html#json>`_\ " which formats proper
json content (but without a field header)
- "`jsonf <property_replacer.html#jsonf>`_\ " which formats as a
complete json field
- "`jsonr <property_replacer.html#jsonr>`_\ " which avoids double
escaping the value but makes it safe for a json field
- "`jsonfr <property_replacer.html#jsonfr>`_\ " which is the
combination of "jsonf" and "jsonr".
- **position.from** - obtain substring starting from this position (1 is
the first position)
- **position.to** - obtain substring up to this position. As a special
extension available since 8.2302.0. the `position.to` value may be
negative (-n). In this case, all characters from `position.from` to the
end of the actual property string except the last n characters are extracted.
This makes it easy to strip the beginning and end of a string with changing
legth. Let us assume you have a string `"[abc]"` and want to remove the
braces. In this case, use `position.from="2" position.to="-1"` in the
`property()` object. This will result in the string `"abc"` after template
processing. This is especially useful if you want to drop the braces around
the `STRUCTURED-DATA` message property.
- **position.relativeToEnd** - the from and to position is relative to the
end of the string instead of the usual start of string. (available
since rsyslog v7.3.10)
- **fixedwidth** - changes behaviour of position.to so that it pads the
source string with spaces up to the value of position.to if the source
string is shorter. "on" or "off" (default) (available since rsyslog
v8.13.0)
- **compressspace** - compresses multiple spaces (US-ASCII SP character)
inside the string to a single one. This compression happens at a very
late stage in processing. Most importantly, it happens after substring
extraction, so the **position.from** and **position.to** positions
are **NOT** affected by this option. (available since v8.18.0).
- **field.number** - obtain this field match
- **field.delimiter** - decimal value of delimiter character for field
extraction
- **regex.expression** - expression to use
- **regex.type** - either ERE or BRE
- **regex.nomatchmode** - what to do if we have no match
- **regex.match** - match to use
- **regex.submatch** - submatch to use
- **droplastlf** - drop a trailing LF, if it is present
- **mandatory** - signifies a field as mandatory. If set to "on", this
field will always be present in data passed to structured outputs,
even if it is empty. If "off" (the default) empty fields will not be
passed to structured outputs. This is especially useful for outputs
that support dynamic schemas (like ommongodb).
- **spifno1stsp** - expert options for RFC3164 template processing
- **datatype** - for "jsonf" format ONLY; permits to set a datatype
Log messages as string data types natively. Thus every property inside
rsyslog is string based. However, in some end systems you need different
data types like numbers of boolean. This setting, in jsonf mode, permits
to configure a desired data type. Supported data types are:
- number - value is treated as a JSON number and not enclosed in quotes.
If the property is empty, the value 0 is generated.
- string - value is a string and enclosed in quotes
- auto - value is treated as number if numeric and as string otherwise.
The current implementation treats only integers as numeric to avoid
confusion.
- bool - the value is treated as boolean. If it is empty or 0, it will
generate "false", else "true".
If not specified, 'string' datatype is assumed.
This is a feature of rsyslog 8.1905.0 or later.
- **onEmpty** - for "jsonf" format ONLY; specifies how empty values
shall be handled. Possible values are:
- keep - emit the empty element
- skip - completely ignore the element, do not emit anything
- null - emit a JSON 'null' value
If not specified, 'keep' is assumed.
This is a feature of rsyslog 8.1905.0 or later.
Subtree
-------
Available since rsyslog 7.1.4
In this case, the template is generated based on a complete (CEE)
subtree. This type of template is most useful for outputs that know how
to process hierarchical structure, like ommongodb. With that type, the
parameter **subtree** must be specified, which tells which subtree to
use. For example template(name="tpl1" type="subtree" subtree="$!")
includes all CEE data, while template(name="tpl2" type="subtree"
subtree="$!usr!tpl2") includes only the subtree starting at $!usr!tpl2.
The core idea when using this type of template is that the actual data
is prefabricated via set and unset script statements, and the resulting
structure is then used inside the template. This method MUST be used if
a complete subtree needs to be placed *directly* into the object's root.
With all other template types, only subcontainers can be generated. Note
that subtree type can also be used with text-based outputs, like omfile.
HOWEVER, you do not have any capability to specify constant text, and as
such cannot include line breaks. As a consequence, using this template
type for text outputs is usually only useful for debugging or very
special cases (e.g. where the text is interpreted by a JSON parser later
on).
Use case
^^^^^^^^
A typical use case is to first create a custom subtree and then include
it into the template, like in this small example:
.. code-block:: none
set $!usr!tpl2!msg = $msg;
set $!usr!tpl2!dataflow = field($msg, 58, 2);
template(name="tpl2" type="subtree" subtree="$!usr!tpl2")
Here, we assume that $msg contains various fields, and the data from a
field is to be extracted and stored - together with the message - as
field content.
String
------
This closely resembles the legacy template statement. It has a mandatory
parameter **string**, which holds the template string to be applied. A
template string is a mix of constant text and replacement variables (see
property replacer). These variables are taken from message or other
dynamic content when the final string to be passed to a plugin is
generated. String-based templates are a great way to specify textual
content, especially if no complex manipulation to properties is
necessary.
This is a sample for a string-based template:
.. code-block:: none
template(name="tpl3" type="string"
string="%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"
)
The text between percent signs ('%') is interpreted by the rsyslog
:doc:`property replacer <property_replacer>`. In a nutshell,
it contains the property to use as well as options for formatting
and further processing. This is very similar to what the ``property``
object in list templates does (it actually is just a different language to
express most of the same things).
Everything outside of the percent signs is constant text. In the
above case, we have mostly spaces between the property values. At the
end of the string, an escape sequence is used.
Escape sequences permit to specify nonprintable characters. They work
very similar to escape sequences in C and many other languages. They
are initiated by the backslash characters and followed by one or more
characters that specify the actual character. For example \\7 is the
US-ASCII BEL character and \\n is a newline. The set is similar to
what C and perl support, but a bit more limited.
Plugin
------
In this case, the template is generated by a plugin (which is then
called a "strgen" or "string generator"). The format is fixed as it is
coded. While this is inflexible, it provides superior performance, and
is often used for that reason (not that "regular" templates are slow -
but in very demanding environments that "last bit" can make a
difference). Refer to the plugin's documentation for further details.
For this type, the parameter **plugin** must be specified and must
contain the name of the plugin as it identifies itself. Note that the
plugin must be loaded prior to being used inside a template.
Config example:
``template(name="tpl4" type="plugin" plugin="mystrgen")``
Options
^^^^^^^
The <options> part is optional. It carries options influencing the
template as a whole and is a part of the template parameters. See details
below. Be sure NOT to mistake template options with property options -
the latter ones are processed by the property replacer and apply to a
SINGLE property, only (and not the whole template).
Template options are case-insensitive. Currently defined are:
**option.sql** - format the string suitable for a SQL statement in MySQL
format. This will replace single quotes ("'") and the backslash
character by their backslash-escaped counterpart ("\\'" and "\\\\")
inside each field. Please note that in MySQL configuration, the
``NO_BACKSLASH_ESCAPES`` mode must be turned off for this format to work
(this is the default).
**option.stdsql** - format the string suitable for a SQL statement that
is to be sent to a standards-compliant sql server. This will replace
single quotes ("'") by two single quotes ("''") inside each field. You
must use stdsql together with MySQL if in MySQL configuration the
``NO_BACKSLASH_ESCAPES`` is turned on.
**option.json** - format the string suitable for a json statement. This
will replace single quotes ("'") by two single quotes ("''") inside each
field.
**option.jsonf** - format the string as JSON object. This means a leading
and trailing curly brace "{" will be added as well as a comma between all
non-terminal properties and constants.
**option.casesensitive** - treat property name references as case
sensitive. The default is "off", where all property name references are
first converted to lowercase during template definition. With this
option turned "on", property names are looked up as defined in the
template. Use this option if you have JSON (``$!*``), local (``!.*``),
or global (``$!\\*``) properties which contain uppercase letters. The
normal Rsyslog properties are case-insensitive, so this option is not
needed for properly referencing those properties.
Use of the options **option.sql**, **option.stdsql**, and
**option.json** are mutually exclusive. Using more than one at the same
time can cause unpredictable behaviour.
Either the **sql** or **stdsql** option **must** be specified when a
template is used for writing to a database, otherwise injection might
occur. Please note that due to the unfortunate fact that several vendors
have violated the sql standard and introduced their own escape methods,
it is impossible to have a single option doing all the work. So you
yourself must make sure you are using the right format. **If you choose
the wrong one, you are still vulnerable to sql injection.**
Please note that the database writer *checks* that the sql option is
present in the template. If it is not present, the write database action
is disabled. This is to guard you against accidentally forgetting it and
then becoming vulnerable to SQL injection. The sql option can also be
useful with files - especially if you want to import them into a
database on another machine for performance reasons. However, do NOT use
it if you do not have a real need for it - among others, it takes some
toll on the processing time. Not much, but on a really busy system you
might notice it.
The default template for the write to database action has the sql option
set. As we currently support only MySQL and the sql option matches the
default MySQL configuration, this is a good choice. However, if you have
turned on ``NO_BACKSLASH_ESCAPES`` in your MySQL config, you need to
supply a template with the stdsql option. Otherwise you will become
vulnerable to SQL injection.
.. code-block:: none
template (name="TraditionalFormat" type="string"
string="%timegenerated% %HOSTNAME% %syslogtag%%msg%\\n")
Examples
========
Standard Template for Writing to Files
--------------------------------------
.. code-block:: none
template(name="FileFormat" type="list") {
property(name="timestamp" dateFormat="rfc3339")
constant(value=" ")
property(name="hostname")
constant(value=" ")
property(name="syslogtag")
property(name="msg" spifno1stsp="on" )
property(name="msg" droplastlf="on" )
constant(value="\n")
}
The equivalent string template looks like this:
.. code-block:: none
template(name="FileFormat" type="string"
string= "%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"
)
.. note::
The template string itself must be on a single line.
Standard Template for Forwarding to a Remote Host (RFC3164 mode)
----------------------------------------------------------------
.. code-block:: none
template(name="ForwardFormat" type="list") {
constant(value="<")
property(name="pri")
constant(value=">")
property(name="timestamp" dateFormat="rfc3339")
constant(value=" ")
property(name="hostname")
constant(value=" ")
property(name="syslogtag" position.from="1" position.to="32")
property(name="msg" spifno1stsp="on" )
property(name="msg")
}
The equivalent string template looks like this:
.. code-block:: none
template(name="forwardFormat" type="string"
string="<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
)
.. note::
The template string itself must be on a single line.
Standard Template for writing to the MySQL database
---------------------------------------------------
.. code-block:: none
template(name="StdSQLformat" type="list" option.sql="on") {
constant(value="insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag)")
constant(value=" values ('")
property(name="msg")
constant(value="', ")
property(name="syslogfacility")
constant(value=", '")
property(name="hostname")
constant(value="', ")
property(name="syslogpriority")
constant(value=", '")
property(name="timereported" dateFormat="mysql")
constant(value="', '")
property(name="timegenerated" dateFormat="mysql")
constant(value="', ")
property(name="iut")
constant(value=", '")
property(name="syslogtag")
constant(value="')")
}
The equivalent string template looks like this:
.. code-block:: none
template(name="stdSQLformat" type="string" option.sql="on"
string="insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')"
)
.. note::
The template string itself must be on a single line.
Generating JSON
---------------
This is especially useful for RESTful APIs, like for example ElasticSearch provides.
This template
.. code-block:: none
template(name="outfmt" type="list" option.jsonf="on") {
property(outname="@timestamp" name="timereported" dateFormat="rfc3339" format="jsonf")
property(outname="host" name="hostname" format="jsonf")
property(outname="severity" name="syslogseverity" caseConversion="upper" format="jsonf" datatype="number")
property(outname="facility" name="syslogfacility" format="jsonf" datatype="number")
property(outname="syslog-tag" name="syslogtag" format="jsonf")
property(outname="source" name="app-name" format="jsonf" onEmpty="null")
property(outname="message" name="msg" format="jsonf")
}
Generates output similar to this
.. code-block:: none
{"@timestamp":"2018-03-01T01:00:00+00:00", "host":"172.20.245.8", "severity":7, "facility":20, "syslog-tag":"tag", "source":"tag", "message":" msgnum:00000000:"}
Pretty-printed this looks like
.. code-block:: none
{
"@timestamp": "2018-03-01T01:00:00+00:00",
"host": "172.20.245.8",
"severity": 7,
"facility": 20,
"syslog-tag": "tag",
"source": "tag",
"message": " msgnum:00000000:"
}
.. note::
The output is **not** pretty-printed as this is just waste of resources when
used in RESTful APIs.
If the "app-name" property is empty, a JSON null value is generated as the `onEmpty="null"`
parameter is used
.. code-block:: none
{"@timestamp":"2018-03-01T01:00:00+00:00", "host":"172.20.245.8", "severity":7, "facility":20, "syslog-tag":"tag", "source":null, "message":" msgnum:00000000:"}
Creating Dynamic File Names for omfile
--------------------------------------
Templates can be used to generate actions with dynamic file names.
For example, if you would like to split syslog messages from different hosts
to different files (one per host), you can define the following template:
.. code-block:: none
template (name="DynFile" type="string" string="/var/log/system-%HOSTNAME%.log")
Reserved Template Names
-----------------------
Template names beginning with "RSYSLOG\_" are reserved for rsyslog use.
Do NOT use them, otherwise you may cause conflicts in the future
(and quite unpredictable behaviour). There is a small set of pre-defined
templates that you can use without the need to define them:
**RSYSLOG_TraditionalFileFormat** - The "old style" default log file
format with low-precision timestamps.
.. code-block:: none
template(name="RSYSLOG_TraditionalFileFormat" type="string"
string="%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n")
**RSYSLOG_FileFormat** - A modern-style logfile format similar to
TraditionalFileFormat, both with high-precision timestamps and
timezone information.
.. code-block:: none
template(name="RSYSLOG_FileFormat" type="list") {
property(name="timereported" dateFormat="rfc3339")
constant(value=" ")
property(name="hostname")
constant(value=" ")
property(name="syslogtag")
property(name="msg" spifno1stsp="on")
property(name="msg" droplastlf="on")
constant(value="\n")
}
**RSYSLOG_TraditionalForwardFormat** - The traditional forwarding format
with low-precision timestamps. Most useful if you send messages to
other syslogd's or rsyslogd below version 3.12.5.
.. code-block:: none
template(name="RSYSLOG_TraditionalForwardFormat" type="string"
string="<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%")
**RSYSLOG_SysklogdFileFormat** - Sysklogd compatible log file format. If
used with options: ``$SpaceLFOnReceive on``,
``$EscapeControlCharactersOnReceive off``, ``$DropTrailingLFOnReception off``,
the log format will conform to sysklogd log format.
.. code-block:: none
template(name="RSYSLOG_SysklogdFileFormat" type="string"
string="%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg%\n")
**RSYSLOG_ForwardFormat** - a new high-precision forwarding format very
similar to the traditional one, but with high-precision timestamps
and timezone information. Recommended to be used when sending
messages to rsyslog 3.12.5 or above.
.. code-block:: none
template(name="RSYSLOG_ForwardFormat" type="string"
string="<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%")
**RSYSLOG_SyslogProtocol23Format** - the format specified in IETF's
internet-draft ietf-syslog-protocol-23, which is very close to the actual
syslog standard `RFC5424 <https://tools.ietf.org/html/rfc5424>`_ (we couldn't
update this template as things were in production for quite some time when
RFC5424 was finally approved). This format includes several improvements.
You may use this format with all relatively recent versions of rsyslog or syslogd.
.. code-block:: none
template(name="RSYSLOG_SyslogProtocol23Format" type="string"
string="<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n")
**RSYSLOG_DebugFormat** - a special format used for troubleshooting
property problems. This format is meant to be written to a log file.
Do **not** use for production or remote forwarding.
.. code-block:: none
template(name="RSYSLOG_DebugFormat" type="list") {
constant(value="Debug line with all properties:\nFROMHOST: '")
property(name="fromhost")
constant(value="', fromhost-ip: '")
property(name="fromhost-ip")
constant(value="', HOSTNAME: '")
property(name="hostname")
constant(value="', PRI: '")
property(name="pri")
constant(value=",\nsyslogtag '")
property(name="syslogtag")
constant(value="', programname: '")
property(name="programname")
constant(value="', APP-NAME: '")
property(name="app-name")
constant(value="', PROCID: '")
property(name="procid")
constant(value="', MSGID: '")
property(name="msgid")
constant(value="',\nTIMESTAMP: '")
property(name="timereported")
constant(value="', STRUCTURED-DATA: '")
property(name="structured-data")
constant(value="',\nmsg: '")
property(name="msg")
constant(value="'\nescaped msg: '")
property(name="msg" controlcharacters="drop")
constant(value="'\ninputname: ")
property(name="inputname")
constant(value=" rawmsg: '")
property(name="rawmsg")
constant(value="'\n$!:")
property(name="$!")
constant(value="\n$.:")
property(name="$.")
constant(value="\n$/:")
property(name="$/")
constant(value="\n\n")
}
**RSYSLOG_WallFmt** - Contains information about the host and the time the
message was generated and at the end the syslogtag and message itself.
.. code-block:: none
template(name="RSYSLOG_WallFmt" type="string"
string="\r\n\7Message from syslogd@%HOSTNAME% at %timegenerated% ...\r\n%syslogtag%%msg%\n\r")
**RSYSLOG_StdUsrMsgFmt** - The syslogtag followed by the message is returned.
.. code-block:: none
template(name="RSYSLOG_StdUsrMsgFmt" type="string"
string=" %syslogtag%%msg%\n\r")
**RSYSLOG_StdDBFmt** - Generates a insert command with the message
properties, into table SystemEvents for a mysql database.
.. code-block:: none
template(name="RSYSLOG_StdDBFmt" type="list") {
constant(value="insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag)")
constant(value=" values ('")
property(name="msg")
constant(value="', ")
property(name="syslogfacility")
constant(value=", '")
property(name="hostname")
constant(value="', ")
property(name="syslogpriority")
constant(value=", '")
property(name="timereported" dateFormat="date-mysql")
constant(value="', '")
property(name="timegenerated" dateFormat="date-mysql")
constant(value="', ")
property(name="iut")
constant(value=", '")
property(name="syslogtag")
constant(value="')")
}
**RSYSLOG_StdPgSQLFmt** - Generates a insert command with the message
properties, into table SystemEvents for a pgsql database.
.. code-block:: none
template(name="RSYSLOG_StdPgSQLFmt" type="string"
string="insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime,
ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%',
%syslogpriority%, '%timereported:::date-pgsql%', '%timegenerated:::date-pgsql%', %iut%,
'%syslogtag%')")
**RSYSLOG_spoofadr** - Generates a message containing nothing more than
the ip address of the sender.
.. code-block:: none
template(name="RSYSLOG_spoofadr" type="string" string="%fromhost-ip%")
**RSYSLOG_StdJSONFmt** - Generates a JSON structure containing the message
properties.
.. code-block:: none
template(name="RSYSLOG_StdJSONFmt" type="string"
string="{\"message\":\"%msg:::json%\",\"fromhost\":\"%HOSTNAME:::json%\",\"facility\":
\"%syslogfacility-text%\",\"priority\":\"%syslogpriority-text%\",\"timereported\":
\"%timereported:::date-rfc3339%\",\"timegenerated\":
\"%timegenerated:::date-rfc3339%\"}")
The LEGACY \$template statement
===============================
Legacy format provides limited functionality, but is still frequently used.
So you probably need to at least be able to understand them. Legacy format
is also good for fairly simple templates.
In legacy, only string templates are supported. The full format is as follows:
.. code-block:: none
$template myname,<string-template>
Here, *myname* is the name of the template (like the *name="myname"* property in
modern template format). The *<string-template>* is exactly the *string* property
like in modern template format.
Take for example the following modern template object:
.. code-block:: none
template(name="tpl3" type="string"
string="%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"
)
It translates to the following legacy template statement:
.. code-block:: none
$template tpl3,"%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n"
See Also
--------
- `How to bind a
template <http://www.rsyslog.com/how-to-bind-a-template/>`_
- `Adding the BOM to a
message <http://www.rsyslog.com/adding-the-bom-to-a-message/>`_
- `How to separate log files by host name of the sending
device <http://www.rsyslog.com/article60/>`_
|