UDP
The udp input plugin allows to retrieve structured JSON or raw messages over a UDP network interface (UDP port).
The plugin supports the following configuration parameters:
Key | Description | Default |
|---|---|---|
Listen | Listener network interface. | 0.0.0.0 |
Port | UDP port where listening for connections | 5170 |
Buffer_Size | Specify the maximum buffer size in KB to receive a JSON message. If not set, the default size will be the value of Chunk_Size. | |
Chunk_Size | By default the buffer to store the incoming JSON messages, do not allocate the maximum memory allowed, instead it allocate memory when is required. The rounds of allocations are set by Chunk_Size in KB. If not set, Chunk_Size is equal to 32 (32KB). | 32 |
Format | Specify the expected payload format. It support the options json and none. When using json, it expects JSON maps, when is set to none, it will split every record using the defined Separator (option below). | json |
Separator | When the expected Format is set to none, Fluent Bit needs a separator string to split the records. By default it uses the breakline character (LF or 0x10). | |
Source_Address_Key | Specify the key where the source address will be injected. | |
In order to receive JSON messages over UDP, you can run the plugin from the command line or through the configuration file:
From the command line you can let Fluent Bit listen for JSON messages with the following options:
$ fluent-bit -i udp -o stdout
By default the service will listen an all interfaces (0.0.0.0) through UDP port 5170, optionally you can change this directly, e.g:
$ fluent-bit -i udp -pport=9090 -o stdout
In the example the JSON messages will only arrive through network interface under 192.168.3.2 address and UDP Port 9090.
In your main configuration file append the following Input & Output sections:
[INPUT]
Name udp
Listen 0.0.0.0
Port 5170
Chunk_Size 32
Buffer_Size 64
Format json
[OUTPUT]
Name stdout
Match *
Once Fluent Bit is running, you can send some messages using the netcat:
$ echo '{"key 1": 123456789, "key 2": "abcdefg"}' | nc -u 127.0.0.1 5170
$ bin/fluent-bit -i udp -o stdout -f 1
Fluent Bit v2.x.x
* Copyright (C) 2015-2022 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io
[2023/07/21 13:01:03] [ info] [fluent bit] version=2.1.7, commit=2474ccc759, pid=9677
[2023/07/21 13:01:03] [ info] [storage] ver=1.2.0, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/07/21 13:01:03] [ info] [cmetrics] version=0.6.3
[2023/07/21 13:01:03] [ info] [ctraces ] version=0.3.1
[2023/07/21 13:01:03] [ info] [input:udp:udp.0] initializing
[2023/07/21 13:01:03] [ info] [input:udp:udp.0] storage_strategy='memory' (memory only)
[2023/07/21 13:01:03] [ info] [output:stdout:stdout.0] worker #0 started
[2023/07/21 13:01:03] [ info] [sp] stream processor started
[0] udp.0: [[1689912069.078189000, {}], {"key 1"=>123456789, "key 2"=>"abcdefg"}]
When receiving payloads in JSON format, there are high performance penalties. Parsing JSON is a very expensive task so you could expect your CPU usage increase under high load environments.
To get faster data ingestion, consider to use the option
Format none to avoid JSON parsing if not needed.Last modified 1mo ago