TCP

The tcp input plugin allows to retrieve structured JSON or raw messages over a TCP network interface (TCP port).

Configuration Parameters

The plugin supports the following configuration parameters:

Getting Started

In order to receive JSON messages over TCP, you can run the plugin from the command line or through the configuration file:

Command Line

From the command line you can let Fluent Bit listen for JSON messages with the following options:

$ fluent-bit -i tcp -o stdout

By default the service will listen an all interfaces (0.0.0.0) through TCP port 5170, optionally you can change this directly, e.g:

$ fluent-bit -i tcp://192.168.3.2:9090 -o stdout

In the example the JSON messages will only arrive through network interface under 192.168.3.2 address and TCP Port 9090.

Configuration File

In your main configuration file append the following Input & Output sections:

[INPUT]
    Name        tcp
    Listen      0.0.0.0
    Port        5170
    Chunk_Size  32
    Buffer_Size 64
    Format      json

[OUTPUT]
    Name        stdout
    Match       *

pipeline:
    inputs:
        - name: tcp
          listen: 0.0.0.0
          port: 5170
          chunk_size: 32
          buffer_size: 64
          format: json
    outputs:
        - name: stdout
          match: '*'

Testing

Once Fluent Bit is running, you can send some messages using the netcat:

$ echo '{"key 1": 123456789, "key 2": "abcdefg"}' | nc 127.0.0.1 5170

In Fluent Bit we should see the following output:

$ bin/fluent-bit -i tcp -o stdout -f 1
Fluent Bit v1.x.x
* Copyright (C) 2019-2020 The Fluent Bit Authors
* Copyright (C) 2015-2018 Treasure Data
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2019/10/03 09:19:34] [ info] [storage] initializing...
[2019/10/03 09:19:34] [ info] [storage] in-memory
[2019/10/03 09:19:34] [ info] [engine] started (pid=14569)
[2019/10/03 09:19:34] [ info] [in_tcp] binding 0.0.0.0:5170
[2019/10/03 09:19:34] [ info] [sp] stream processor started
[0] tcp.0: [1570115975.581246030, {"key 1"=>123456789, "key 2"=>"abcdefg"}]

Performance Considerations

When receiving payloads in JSON format, there are high performance penalties. Parsing JSON is a very expensive task so you could expect your CPU usage increase under high load environments.

To get faster data ingestion, consider to use the option Format none to avoid JSON parsing if not needed.