Comment on page

Syslog

Syslog input plugins allows to collect Syslog messages through a Unix socket server (UDP or TCP) or over the network using TCP or UDP.
Configuration Parameters
The plugin supports the following configuration parameters:
Key
Description
Default
Mode
Defines transport protocol mode: unix_udp (UDP over Unix socket), unix_tcp (TCP over Unix socket), tcp or udp
unix_udp
Listen
If Mode is set to tcp or udp, specify the network interface to bind.
0.0.0.0
Port
If Mode is set to tcp or udp, specify the TCP port to listen for incoming connections.
5140
Path
If Mode is set to unix_tcp or unix_udp, set the absolute path to the Unix socket file.
​
Unix_Perm
If Mode is set to unix_tcp or unix_udp, set the permission of the Unix socket file.
0644
Parser
Specify an alternative parser for the message. If Mode is set to tcp or udp then the default parser is syslog-rfc5424 otherwise syslog-rfc3164-local is used. If your syslog messages have fractional seconds set this Parser value to syslog-rfc5424 instead.
​
Buffer_Chunk_Size
By default the buffer to store the incoming Syslog messages, do not allocate the maximum memory allowed, instead it allocate memory when is required. The rounds of allocations are set by Buffer_Chunk_Size. If not set, Buffer_Chunk_Size is equal to 32000 bytes (32KB). Read considerations below when using udp or unix_udp mode.
​
Buffer_Max_Size
Specify the maximum buffer size to receive a Syslog message. If not set, the default size will be the value of Buffer_Chunk_Size.
​
Receive_Buffer_Size
Specify the maximum socket receive buffer size. If not set, the default value is OS-dependant, but generally too low to accept thousands of syslog messages per second without loss on udp or unix_udp sockets. Note that on Linux the value is capped by sysctl net.core.rmem_max.
​
Source_Address_Key
Specify the key where the source address will be injected.
​
Considerations
When using Syslog input plugin, Fluent Bit requires access to the parsers.conf file, the path to this file can be specified with the option -R or through the Parsers_File key on the [SERVICE] section (more details below).
When udp or unix_udp is used, the buffer size to receive messages is configurable only through the Buffer_Chunk_Size option which defaults to 32kb.
Getting Started
In order to receive Syslog messages, you can run the plugin from the command line or through the configuration file:
Command Line
From the command line you can let Fluent Bit listen for Forward messages with the following options:
$ fluent-bit -R /path/to/parsers.conf -i syslog -p path=/tmp/in_syslog -o stdout
By default the service will create and listen for Syslog messages on the unix socket /tmp/in_syslog
Configuration File
In your main configuration file append the following Input & Output sections:
[SERVICE]
    Flush               1
    Log_Level           info
    Parsers_File        parsers.conf
​
[INPUT]
    Name                syslog
    Path                /tmp/in_syslog
    Buffer_Chunk_Size   32000
    Buffer_Max_Size     64000
    Receive_Buffer_Size 512000
​
[OUTPUT]
    Name   stdout
    Match  *
Testing
Once Fluent Bit is running, you can send some messages using the logger tool:
$ logger -u /tmp/in_syslog my_ident my_message
In Fluent Bit we should see the following output:
$ bin/fluent-bit -R ../conf/parsers.conf -i syslog -p path=/tmp/in_syslog -o stdout
Fluent Bit v1.x.x
* Copyright (C) 2019-2020 The Fluent Bit Authors
* Copyright (C) 2015-2018 Treasure Data
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io
​
[2017/03/09 02:23:27] [ info] [engine] started
[0] syslog.0: [1489047822, {"pri"=>"13", "host"=>"edsiper:", "ident"=>"my_ident", "pid"=>"", "message"=>"my_message"}]
Recipes
The following content aims to provide configuration examples for different use cases to integrate Fluent Bit and make it listen for Syslog messages from your systems.
Rsyslog to Fluent Bit: Network mode over TCP
Fluent Bit Configuration
Put the following content in your fluent-bit.conf file:
[SERVICE]
    Flush        1
    Parsers_File parsers.conf
​
[INPUT]
    Name     syslog
    Parser   syslog-rfc3164
    Listen   0.0.0.0
    Port     5140
    Mode     tcp
​
[OUTPUT]
    Name     stdout
    Match    *
then start Fluent Bit.
RSyslog Configuration
Add a new file to your rsyslog config rules called 60-fluent-bit.conf inside the directory /etc/rsyslog.d/ and add the following content:
action(type="omfwd" Target="127.0.0.1" Port="5140" Protocol="tcp")
then make sure to restart your rsyslog daemon:
$ sudo service rsyslog restart
Rsyslog to Fluent Bit: Unix socket mode over UDP
Fluent Bit Configuration
Put the following content in your fluent-bit.conf file:
[SERVICE]
    Flush        1
    Parsers_File parsers.conf
​
[INPUT]
    Name      syslog
    Parser    syslog-rfc3164
    Path      /tmp/fluent-bit.sock
    Mode      unix_udp
    Unix_Perm 0644
​
[OUTPUT]
    Name      stdout
    Match     *
then start Fluent Bit.
RSyslog Configuration
Add a new file to your rsyslog config rules called 60-fluent-bit.conf inside the directory /etc/rsyslog.d/ and place the following content:
$ModLoad omuxsock
$OMUxSockSocket /tmp/fluent-bit.sock
*.* :omuxsock:
Make sure that the socket file is readable by rsyslog (tweak the Unix_Perm option shown above).

Key	Description	Default
Mode	Defines transport protocol mode: unix_udp (UDP over Unix socket), unix_tcp (TCP over Unix socket), tcp or udp	unix_udp
Listen	If Mode is set to tcp or udp, specify the network interface to bind.	0.0.0.0
Port	If Mode is set to tcp or udp, specify the TCP port to listen for incoming connections.	5140
Path	If Mode is set to unix_tcp or unix_udp, set the absolute path to the Unix socket file.
Unix_Perm	If Mode is set to unix_tcp or unix_udp, set the permission of the Unix socket file.	0644
Parser	Specify an alternative parser for the message. If Mode is set to tcp or udp then the default parser is syslog-rfc5424 otherwise syslog-rfc3164-local is used. If your syslog messages have fractional seconds set this Parser value to syslog-rfc5424 instead.
Buffer_Chunk_Size	By default the buffer to store the incoming Syslog messages, do not allocate the maximum memory allowed, instead it allocate memory when is required. The rounds of allocations are set by Buffer_Chunk_Size. If not set, Buffer_Chunk_Size is equal to 32000 bytes (32KB). Read considerations below when using udp or unix_udp mode.
Buffer_Max_Size	Specify the maximum buffer size to receive a Syslog message. If not set, the default size will be the value of Buffer_Chunk_Size.
Receive_Buffer_Size	Specify the maximum socket receive buffer size. If not set, the default value is OS-dependant, but generally too low to accept thousands of syslog messages per second without loss on udp or unix_udp sockets. Note that on Linux the value is capped by `sysctl net.core.rmem_max`.
Source_Address_Key	Specify the key where the source address will be injected.

Last modified 2mo ago