All pages
Powered by GitBook
1 of 4

Loading...

Loading...

Loading...

Loading...

Build with Static Configuration

Fluent Bit in normal operation mode allows to be configurable through text files or using specific arguments in the command line, while this is the ideal deployment case, there are scenarios where a more restricted configuration is required: static configuration mode.

Static configuration mode aims to include a built-in configuration in the final binary of Fluent Bit, disabling the usage of external files or flags at runtime.

Getting Started

Requirements

The following steps assumes you are familiar with configuring Fluent Bit using text files and you have experience building it from scratch as described in the Build and Install section.

Configuration Directory

In your file system prepare a specific directory that will be used as an entry point for the build system to lookup and parse the configuration files. It is mandatory that this directory contain as a minimum one configuration file called fluent-bit.conf containing the required SERVICE, INPUT and OUTPUT sections. As an example create a new fluent-bit.conf file with the following content:

[SERVICE]
    Flush     1
    Daemon    off
    Log_Level info

[INPUT]
    Name      cpu

[OUTPUT]
    Name      stdout
    Match     *

the configuration provided above will calculate CPU metrics from the running system and print them to the standard output interface.

Build with Custom Configuration

Inside Fluent Bit source code, get into the build/ directory and run CMake appending the FLB_STATIC_CONF option pointing the configuration directory recently created, e.g:

$ cd fluent-bit/build/
$ cmake -DFLB_STATIC_CONF=/path/to/my/confdir/

then build it:

$ make

At this point the fluent-bit binary generated is ready to run without necessity of further configuration:

$ bin/fluent-bit 
Fluent-Bit v0.15.0
Copyright (C) Treasure Data

[2018/10/19 15:32:31] [ info] [engine] started (pid=15186)
[0] cpu.local: [1539984752.000347547, {"cpu_p"=>0.750000, "user_p"=>0.500000, "system_p"=>0.250000, "cpu0.p_cpu"=>1.000000, "cpu0.p_user"=>1.000000, "cpu0.p_system"=>0.000000, "cpu1.p_cpu"=>0.000000, "cpu1.p_user"=>0.000000, "cpu1.p_system"=>0.000000, "cpu2.p_cpu"=>0.000000, "cpu2.p_user"=>0.000000, "cpu2.p_system"=>0.000000, "cpu3.p_cpu"=>1.000000, "cpu3.p_user"=>1.000000, "cpu3.p_system"=>0.000000}]

Download Source Code

Stable

For production systems, we strongly suggest that you always get the latest stable release of the source code in either zip or tarball format from Github using the following link pattern:

https://github.com/fluent/fluent-bit/archive/refs/tags/v<release version>.tar.gz https://github.com/fluent/fluent-bit/archive/refs/tags/v<release version>.zip

For example for version 1.8.12 the link is the following: https://github.com/fluent/fluent-bit/archive/refs/tags/v1.8.12.tar.gz

Development

For anyone who aims to contribute to the project by testing or extending the code base, you can get the development version from our GIT repository:

$ git clone https://github.com/fluent/fluent-bit

Note that our master branch is where the development of Fluent Bit happens. Since it's a development version, expect issues when compiling or at run time.

We encourage everybody to help us testing every development version, at the end this is what will become stable.

Sources

Build and Install

Fluent Bit uses CMake as it build system. The suggested procedure to prepare the build system consists of the following steps:

Prepare environment

In the following steps you can find exact commands to build and install the project with the default options. If you already know how CMake works you can skip this part and look at the build options available. Note that Fluent Bit requires CMake 3.x. You may need to use cmake3 instead of cmake to complete the following steps on your system.

Change to the build/ directory inside the Fluent Bit sources:

$ cd build/

Let CMake configure the project specifying where the root path is located:

$ cmake ../
-- The C compiler identification is GNU 4.9.2
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- The CXX compiler identification is GNU 4.9.2
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
...
-- Could NOT find Doxygen (missing:  DOXYGEN_EXECUTABLE)
-- Looking for accept4
-- Looking for accept4 - not found
-- Configuring done
-- Generating done
-- Build files have been written to: /home/edsiper/coding/fluent-bit/build

Now you are ready to start the compilation process through the simple make command:

$ make
Scanning dependencies of target msgpack
[  2%] Building C object lib/msgpack-1.1.0/CMakeFiles/msgpack.dir/src/unpack.c.o
[  4%] Building C object lib/msgpack-1.1.0/CMakeFiles/msgpack.dir/src/objectc.c.o
[  7%] Building C object lib/msgpack-1.1.0/CMakeFiles/msgpack.dir/src/version.c.o
...
[ 19%] Building C object lib/monkey/mk_core/CMakeFiles/mk_core.dir/mk_file.c.o
[ 21%] Building C object lib/monkey/mk_core/CMakeFiles/mk_core.dir/mk_rconf.c.o
[ 23%] Building C object lib/monkey/mk_core/CMakeFiles/mk_core.dir/mk_string.c.o
...
Scanning dependencies of target fluent-bit-static
[ 66%] Building C object src/CMakeFiles/fluent-bit-static.dir/flb_pack.c.o
[ 69%] Building C object src/CMakeFiles/fluent-bit-static.dir/flb_input.c.o
[ 71%] Building C object src/CMakeFiles/fluent-bit-static.dir/flb_output.c.o
...
Linking C executable ../bin/fluent-bit
[100%] Built target fluent-bit-bin

to continue installing the binary on the system just do:

$ make install

it's likely you may need root privileges so you can try to prefixing the command with sudo.

Build Options

Fluent Bit provides certain options to CMake that can be enabled or disabled when configuring, please refer to the following tables under the General Options, Development Options, Input Plugins and _Output Plugins sections.

General Options

option
description
default

FLB_ALL

Enable all features available

No

FLB_JEMALLOC

Use Jemalloc as default memory allocator

No

FLB_TLS

Build with SSL/TLS support

Yes

FLB_BINARY

Build executable

Yes

FLB_EXAMPLES

Build examples

Yes

FLB_SHARED_LIB

Build shared library

Yes

FLB_MTRACE

Enable mtrace support

No

FLB_INOTIFY

Enable Inotify support

Yes

FLB_POSIX_TLS

Force POSIX thread storage

No

FLB_SQLDB

Enable SQL embedded database support

No

FLB_HTTP_SERVER

Enable HTTP Server

No

FLB_LUAJIT

Enable Lua scripting support

Yes

FLB_RECORD_ACCESSOR

Enable record accessor

Yes

FLB_SIGNV4

Enable AWS Signv4 support

Yes

FLB_STATIC_CONF

Build binary using static configuration files. The value of this option must be a directory containing configuration files.

FLB_STREAM_PROCESSOR

Enable Stream Processor

Yes

Development Options

option
description
default

FLB_DEBUG

Build binaries with debug symbols

No

FLB_VALGRIND

Enable Valgrind support

No

FLB_TRACE

Enable trace mode

No

FLB_SMALL

Minimise binary size

No

FLB_TESTS_RUNTIME

Enable runtime tests

No

FLB_TESTS_INTERNAL

Enable internal tests

No

FLB_TESTS

Enable tests

No

FLB_BACKTRACE

Enable backtrace/stacktrace support

Yes

Input Plugins

The input plugins provides certain features to gather information from a specific source type which can be a network interface, some built-in metric or through a specific input device, the following input plugins are available:

option
description
default

Enable Collectd input plugin

On

Enable CPU input plugin

On

Enable Disk I/O Metrics input plugin

On

Enable Docker metrics input plugin

On

Enable Exec input plugin

On

Enable Forward input plugin

On

Enable Head input plugin

On

Enable Health input plugin

On

Enable Kernel log input plugin

On

Enable Memory input plugin

On

Enable MQTT Server input plugin

On

Enable Network I/O metrics input plugin

On

Enable Process monitoring input plugin

On

Enable Random input plugin

On

Enable Serial input plugin

On

Enable Standard input plugin

On

Enable Syslog input plugin

On

Enable Systemd / Journald input plugin

On

Enable Tail (follow files) input plugin

On

Enable TCP input plugin

On

Enable system temperature(s) input plugin

On

Enable Windows Event Log input plugin (Windows Only)

On

Filter Plugins

The filter plugins allows to modify, enrich or drop records. The following table describes the filters available on this version:

option
description
default

Enable AWS metadata filter

On

FLB_FILTER_EXPECT

Enable Expect data test filter

On

Enable Grep filter

On

Enable Kubernetes metadata filter

On

Enable Lua scripting filter

On

Enable Modify filter

On

Enable Nest filter

On

Enable Parser filter

On

Enable Record Modifier filter

On

Enable Rewrite Tag filter

On

Enable Stdout filter

On

Enable Throttle filter

On

Output Plugins

The output plugins gives the capacity to flush the information to some external interface, service or terminal, the following table describes the output plugins available as of this version:

option
description
default

Enable Microsoft Azure output plugin

On

Enable Google BigQuery output plugin

On

Enable Counter output plugin

On

Enable Amazon CloudWatch output plugin

On

Enable Datadog output plugin

On

Enable output plugin

On

Enable File output plugin

On

Enable Amazon Kinesis Data Firehose output plugin

On

Enable Amazon Kinesis Data Streams output plugin

On

Enable Flowcounter output plugin

On

Enable output plugin

On

Enable Gelf output plugin

On

Enable HTTP output plugin

On

Enable InfluxDB output plugin

On

Enable Kafka output

Off

Enable Kafka REST Proxy output plugin

On

FLB_OUT_LIB

Enable Lib output plugin

On

Enable output plugin

On

FLB_OUT_NULL

Enable NULL output plugin

On

FLB_OUT_PGSQL

Enable PostgreSQL output plugin

On

FLB_OUT_PLOT

Enable Plot output plugin

On

FLB_OUT_SLACK

Enable Slack output plugin

On

Enable Amazon S3 output plugin

On

Enable Splunk output plugin

On

Enable Google Stackdriver output plugin

On

Enable STDOUT output plugin

On

FLB_OUT_TCP

Enable TCP/TLS output plugin

On

Enable output plugin

On

FLB_IN_COLLECTD
FLB_IN_CPU
FLB_IN_DISK
FLB_IN_DOCKER
FLB_IN_EXEC
FLB_IN_FORWARD
FLB_IN_HEAD
FLB_IN_HEALTH
FLB_IN_KMSG
FLB_IN_MEM
FLB_IN_MQTT
FLB_IN_NETIF
FLB_IN_PROC
FLB_IN_RANDOM
FLB_IN_SERIAL
FLB_IN_STDIN
FLB_IN_SYSLOG
FLB_IN_SYSTEMD
FLB_IN_TAIL
FLB_IN_TCP
FLB_IN_THERMAL
FLB_IN_WINLOG
FLB_FILTER_AWS
FLB_FILTER_GREP
FLB_FILTER_KUBERNETES
FLB_FILTER_LUA
FLB_FILTER_MODIFY
FLB_FILTER_NEST
FLB_FILTER_PARSER
FLB_FILTER_RECORD_MODIFIER
FLB_FILTER_REWRITE_TAG
FLB_FILTER_STDOUT
FLB_FILTER_THROTTLE
FLB_OUT_AZURE
FLB_OUT_BIGQUERY
FLB_OUT_COUNTER
FLB_OUT_CLOUDWATCH_LOGS
FLB_OUT_DATADOG
FLB_OUT_ES
Elastic Search
FLB_OUT_FILE
FLB_OUT_KINESIS_FIREHOSE
FLB_OUT_KINESIS_STREAMS
FLB_OUT_FLOWCOUNTER
FLB_OUT_FORWARD
Fluentd
FLB_OUT_GELF
FLB_OUT_HTTP
FLB_OUT_INFLUXDB
FLB_OUT_KAFKA
FLB_OUT_KAFKA_REST
FLB_OUT_NATS
NATS
FLB_OUT_S3
FLB_OUT_SPLUNK
FLB_OUT_STACKDRIVER
FLB_OUT_STDOUT
FLB_OUT_TD
Treasure Data