In 2014, the Fluentd team at Treasure Data was forecasting the need for a lightweight log processor for constraint environments like embedded Linux and gateways. The project aimed to be part of the Fluentd ecosystem. At that moment, Eduardo Silva created Fluent Bit, a new open source solution, written from scratch and available under the terms of the Apache License v2.0.

After the project matured, it gained traction for normal Linux systems. With the new containerized world, the Cloud Native community asked to extend the project scope to support more sources, filters, and destinations. Not long after, Fluent Bit became one of the preferred solutions to solve the logging challenges in Cloud environments.

When Fluent Bit processes data, it uses the system memory (heap) as a primary and temporary place to store the record logs before they get delivered. The records are processed in this private memory area.

Buffering is the ability to store the records, and continue storing incoming data while previous data is processed and delivered. Buffering in memory is the fastest mechanism, but there are scenarios requiring special strategies to deal with backpressure, data safety, or to reduce memory consumption by the service in constrained environments.

Network failures or latency in third party service is common. When data can't be delivered fast enough and new data to process arrives, the system can face backpressure.

Fluent Bit buffering strategies are designed to solve problems associated with backpressure and general delivery failures. Fluent Bit offers a primary buffering mechanism in memory and an optional secondary one using the file system. With this hybrid solution you can accommodate any use case safely and keep a high performance while processing your data.

These mechanisms aren't mutually exclusive. When data is ready to be processed or delivered it's always be in memory, while other data in the queue might be in the file system until is ready to be processed and moved up to memory.

To learn more about the buffering configuration in Fluent Bit, see Buffering & Storage.

Fluent Bit is an open source telemetry agent specifically designed to efficiently handle the challenges of collecting and processing telemetry data across a wide range of environments, from constrained systems to complex cloud infrastructures. Managing telemetry data from various sources and formats can be a constant challenge, particularly when performance is a critical factor.

Rather than serving as a drop-in replacement, Fluent Bit enhances the observability strategy for your infrastructure by adapting and optimizing your existing logging layer, and adding metrics and traces processing. Fluent Bit supports a vendor-neutral approach, seamlessly integrating with other ecosystems such as Prometheus and OpenTelemetry. Trusted by major cloud providers, banks, and companies in need of a ready-to-use telemetry agent solution, Fluent Bit effectively manages diverse data sources and formats while maintaining optimal performance and keeping resource consumption low.

Fluent Bit can be deployed as an edge agent for localized telemetry data handling or utilized as a central aggregator/collector for managing telemetry data across multiple sources and environments.

Telemetry data processing can be complex, especially at scale. That's why Fluentd was created. Fluentd is more than a simple tool, it's grown into a fullscale ecosystem that contains SDKs for different languages and subprojects like Fluent Bit.

Here, we describe the relationship between the Fluentd and Fluent Bit open source projects.

Both projects are:

• Licensed under the terms of Apache License v2.0.

• Graduated hosted projects by the Cloud Native Computing Foundation (CNCF).

• Production grade solutions: Deployed millions of times every single day.

• Vendor neutral and community driven.

• Widely adopted by the industry: Trusted by major companies like AWS, Microsoft, Google Cloud, and hundreds of others.

The projects have many similarities: Fluent Bit is designed and built on top of the best ideas of Fluentd architecture and general design. Which one you choose depends on your end-users' needs.

The following table describes a comparison of different areas of the projects:

Both Fluentd and Fluent Bit can work as Aggregators or Forwarders, and can complement each other or be used as standalone solutions.

In the recent years, cloud providers have switched from Fluentd to Fluent Bit for performance and compatibility. Fluent Bit is now considered the next-generation solution.

Dealing with raw strings or unstructured messages is difficult. Having a structure makes data more usable. Set a structure to the incoming data by using input plugins as data is collected:

The parser converts unstructured data to structured data. As an example, consider the following Apache (HTTP Server) log entry:

192.168.2.20 - - [28/Jul/2006:10:27:10 -0300] "GET /cgi-bin/try/ HTTP/1.0" 200 3395

This log line is a raw string without format. Structuring the log makes it easier to process the data later. If the regular expression parser is used, the log entry could be converted to:

{
  "host":    "192.168.2.20",
  "user":    "-",
  "method":  "GET",
  "path":    "/cgi-bin/try/",
  "code":    "200",
  "size":    "3395",
  "referer": "",
  "agent":   ""
 }

Parsers are fully configurable and are independently and optionally handled by each input plugin. For more details, see Parsers.

Fluent Bit provides input plugins to gather information from different sources. Some plugins collect data from log files, while others can gather metrics information from the operating system. There are many plugins to suit different needs.

When an input plugin loads, an internal instance is created. Each instance has its own independent configuration. Configuration keys are often called properties.

Every input plugin has its own documentation section that specifies how to use it and what properties are available.

For more details, see Input Plugins.

In production environments you need full control of the data you're collecting. Filtering lets you alter the collected data before delivering it to a destination.

Filtering is implemented through plugins. Each available filter can be used to match, exclude, or enrich your logs with specific metadata.

Fluent Bit support many filters. A common use case for filtering is Kubernetes deployments. Every pod log needs the proper metadata associated with it.

Like input plugins, filters run in an instance context, which has its own independent configuration. Configuration keys are often called properties.

For more details about the Filters available and their usage, see Filters.

The buffer phase in the pipeline aims to provide a unified and persistent mechanism to store your data, using the primary in-memory model or the file system-based mode.

The buffer phase contains the data in an immutable state, meaning that no other filter can be applied.

Buffered data uses the Fluent Bit internal binary representation, which isn't raw text.

Fluent Bit offers a buffering mechanism in the file system that acts as a backup system to avoid data loss in case of system failures.

Fluent Bit is a fast and lightweight telemetry agent for logs, metrics, and traces for Linux, macOS, Windows, and BSD family operating systems. Fluent Bit has been made with a strong focus on performance to allow the collection and processing of telemetry data from different sources without complexity.

Features

• High performance: High throughput with low resources consumption

• Data parsing

  • Convert your unstructured messages using our parsers: JSON, Regex, LTSV and Logfmt

• Metrics support: Prometheus and OpenTelemetry compatible

• Reliability and data integrity

  • Backpressure handling

  • Data buffering in memory and file system

• Networking

  • Security: Built-in TLS/SSL support

  • Asynchronous I/O

• Pluggable architecture and extensibility: Inputs, Filters and Outputs:

  • Connect nearly any source to nearly any destination using preexisting plugins

  • Extensibility:

    • Write input, filter, or output plugins in the C language

    • WASM: WASM Filter Plugins or WASM Input Plugins

    • Write Filters in Lua or Output plugins in Golang

• Monitoring: Expose internal metrics over HTTP in JSON and Prometheus format

• Stream Processing: Perform data selection and transformation using simple SQL queries

  • Create new streams of data using query results

  • Aggregation windows

  • Data analysis and prediction: Timeseries forecasting

• Portable: Runs on Linux, macOS, Windows and BSD systems

Fluent Bit, Fluentd and CNCF

Fluent Bit is a CNCF graduated sub-project under the umbrella of Fluentd. Fluent Bit is licensed under the terms of the Apache License v2.0.

Fluent Bit was originally created by Eduardo Silva and is now sponsored by Chronosphere. As a CNCF-hosted project, it is a fully vendor-neutral and community-driven project.

The output interface lets you define destinations for your data. Common destinations are remote services, local file systems, or other standard interfaces. Outputs are implemented as plugins.

When an output plugin is loaded, an internal instance is created. Every instance has its own independent configuration. Configuration keys are often called properties.

Every output plugin has its own documentation section specifying how it can be used and what properties are available.

For more details, see .

Routing is a core feature that lets you route your data through filters and then to one or multiple destinations. The router relies on the concept of and rules.

There are two important concepts in Routing:

• Tag

• Match

When data is generated by an input plugin, it comes with a Tag. A Tag is a human-readable indicator that helps to identify the data source. Tags are usually configured manually.

To define where to route data, specify a Match rule in the output configuration.

Consider the following configuration example that delivers CPU metrics to an Elasticsearch database and Memory (mem) metrics to the standard output interface:

Routing reads the Input Tag and the Output Match rules. If data has a Tag that doesn't match at routing time, the data is deleted.

Routing with Wildcard

Routing is flexible enough to support wildcards in the Match pattern. The following example defines a common destination for both sources of data:

The match rule is set to my_*, which matches any Tag starting with my_*.

Routing with Regex

Routing also provides support for regular expressions with the Match_Regex pattern, allowing for more complex and precise matching criteria. The following example demonstrates how to route data from sources based on a regular expression:

In this configuration, the Match_regex rule is set to .*_sensor_[AB]. This regular expression matches any Tag that ends with _sensor_A or _sensor_B, regardless of what precedes it. This approach provides a more flexible and powerful way to handle different source tags with a single routing rule.

Fluent Bit supports the following operating systems and architectures:

From an architecture support perspective, Fluent Bit is fully functional on x86_64, Arm64v8, and Arm32v7 based processors.

Fluent Bit can work also on macOS and Berkeley Software Distribution (BSD) systems, but not all plugins will be available on all platforms.

Fluent Bit is supported for Linux on IBM Z (s390x) environments with some restrictions, but only container images are provided for these targets officially.

Container deployment

Install on Linux (packages)

Install on Windows (packages)

Install on macOS (packages)

Compile from Source (Linux, Windows, FreeBSD, macOS)

Sandbox Environment

If you are interested in learning about Fluent Bit you can try out the sandbox environment:

Enterprise Packages

uses as its build system.

Requirements

• CMake 3.12 or greater. You might need to use cmake3 instead of cmake.

• Flex

• Bison 3 or greater

• YAML headers

• OpenSSL headers

Prepare environment

If you already know how CMake works, you can skip this section and review the available .

The following steps explain how to build and install the project with the default options.

1. Change to the build/ directory inside the Fluent Bit sources:

2. Let configure the project specifying where the root path is located:

   This command displays a series of results similar to:

3. Start the compilation process using the make command:

   This command displays results similar to:

4. To continue installing the binary on the system, use make install:

   If the command indicates insufficient permissions, prefix the command with sudo.

Build options

Fluent Bit provides configurable options to CMake that can be enabled or disabled.

General options

Development options

Optimization options

Input plugins

Input plugins gather information from a specific source type like network interfaces, some built-in metrics, or through a specific input device. The following input plugins are available:

Filter plugins

Filter plugins let you modify, enrich or drop records. The following table describes the filters available on this version:

Output plugins

Output plugins let you flush the information to some external interface, service, or terminal. The following table describes the output plugins available:

Processor plugins

Processor plugins handle the events within the processor pipelines to allow modifying, enriching, or dropping events.

The following table describes the processors available:

Before diving into you might want to get acquainted with some of the key concepts of the service. This document provides an introduction to those concepts and common terminology. Reading this document will help you gain a more general understanding of the following topics:

• Event or Record

• Filtering

• Tag

• Timestamp

• Match

• Structured Message

Event or Record

Every incoming piece of data that belongs to a log or a metric that's retrieved by Fluent Bit is considered an Event or a Record.

As an example, consider the following content of a Syslog file:

It contains four lines that represent four independent Events.

An Event is comprised of:

• timestamp

• key/value metadata (v2.1.0 and greater)

• payload

Event format

The Fluent Bit wire protocol represents an Event as a two-element array with a nested array as the first element:

where

• TIMESTAMP is a timestamp in seconds as an integer or floating point value (not a string).

• METADATA is an object containing event metadata, and might be empty.

• MESSAGE is an object containing the event body.

Fluent Bit versions prior to v2.1.0 used:

to represent events. This format is still supported for reading input event streams.

Filtering

Use filtering to:

• Append specific information to the Event like an IP address or metadata.

• Select a specific piece of the Event content.

• Drop Events that match a certain pattern.

Tag

Timestamp

The timestamp represents the time an Event was created. Every Event contains an associated timestamps. All events have timestamps, and they're set by the input plugin or discovered through a data parsing process.

The timestamp is a numeric fractional integer in the format:

where:

• _SECONDS_ is the number of seconds that have elapsed since the Unix epoch.

• _NANOSECONDS_ is a fractional second or one thousand-millionth of a second.

Match

Fluent Bit lets you route your collected and processed Events to one or multiple destinations. A Match represents a rule to select Events where a Tag matches a defined rule.

Structured messages

Source events can have a structure. A structure defines a set of keys and values inside the Event message to implement faster operations on data modifications. Fluent Bit treats every Event message as a structured message.

Consider the following two messages:

• No structured message
• With a structured message

, including its core, plugins, and tools are distributed under the terms of the :

has very low CPU and memory consumption. It's compatible with most x86-, x86_64-, arm32v7-, and arm64v8-based platforms.

The build process requires the following components:

• Compiler: GCC or clang

• CMake

• Flex and Bison: Required for or

• Libyaml development headers and libraries

Core has no other dependencies. Some features depend on third-party components. For example, output plugins with special backend libraries like Kafka include those libraries in the main source code repository.

Fluent Bit is supported on Linux on IBM Z(s390x), but the WASM and LUA filter plugins aren't.

The following article cover the relevant notes for users upgrading from previous Fluent Bit versions. We aim to cover compatibility changes that you must be aware of.

For more details about changes on each release please refer to the .

Note: release notes will be prepared in advance of a Git tag for a release so an official release should provide both a tag and a release note together to allow users to verify and understand the release contents.

The tag drives the overall binary release process so release binaries (containers/packages) will appear after a tag and its associated release note. This allows users to expect the new release binary to appear and allow/deny/update it as appropriate in their infrastructure.

Fluent Bit v1.9.9

The td-agent-bit package is no longer provided after this release. Users should switch to the fluent-bit package.

Fluent Bit v1.6

If you are migrating from previous version of Fluent Bit please review the following important changes:

Tail Input Plugin

Now by default the plugin follows a file from the end once the service starts (old behavior was always read from the beginning). For every file found at start, its followed from it last position, for new files discovered at runtime or rotated, they are read from the beginning.

If you desire to keep the old behavior you can set the option read_from_head to true.

Stackdriver Output Plugin

The project_id of in sent to Google Cloud Logging would be set to the project ID rather than the project number. To learn the difference between Project ID and project number, see for more details.

If you have any existing queries based on the resource's project_id, please update your query accordingly.

Fluent Bit v1.5

The migration from v1.4 to v1.5 is pretty straightforward.

Fluent Bit v1.4

If you are migrating from Fluent Bit v1.3, there are no breaking changes. Just new exciting features to enjoy :)

Fluent Bit v1.3

If you are migrating from Fluent Bit v1.2 to v1.3, there are no breaking changes. If you are upgrading from an older version please review the incremental changes below.

Fluent Bit v1.2

Docker, JSON, Parsers and Decoders

On Fluent Bit v1.2 we have fixed many issues associated with JSON encoding and decoding, for hence when parsing Docker logs is no longer necessary to use decoders. The new Docker parser looks like this:

Note: again, do not use decoders.

Kubernetes Filter

We have done improvements also on how Kubernetes Filter handle the stringified log message. If the option Merge_Log is enabled, it will try to handle the log content as a JSON map, if so, it will add the keys to the root map.

In addition, we have fixed and improved the option called Merge_Log_Key. If a merge log succeed, all new keys will be packaged under the key specified by this option, a suggested configuration is as follows:

As an example, if the original log content is the following map:

the final record will be composed as follows:

Fluent Bit v1.1

If you are upgrading from Fluent Bit <= 1.0.x you should take in consideration the following relevant changes when switching to Fluent Bit v1.1 series:

Kubernetes Filter

We introduced a new configuration property called Kube_Tag_Prefix to help Tag prefix resolution and address an unexpected behavior that landed in previous versions.

During 1.0.x release cycle, a commit in Tail input plugin changed the default behavior on how the Tag was composed when using the wildcard for expansion generating breaking compatibility with other services. Consider the following configuration example:

The expected behavior is that Tag will be expanded to:

but the change introduced in 1.0 series switched from absolute path to the base file name only:

On Fluent Bit v1.1 release we restored to our default behavior and now the Tag is composed using the absolute path of the monitored file.

Having absolute path in the Tag is relevant for routing and flexible configuration where it also helps to keep compatibility with Fluentd behavior.

This behavior switch in Tail input plugin affects how Filter Kubernetes operates. As you know when the filter is used it needs to perform local metadata lookup that comes from the file names when using Tail as a source. Now with the new Kube_Tag_Prefix option you can specify what's the prefix used in Tail input plugin, for the configuration example above the new configuration will look as follows:

So the proper for Kube_Tag_Prefix value must be composed by Tag prefix set in Tail input plugin plus the converted monitored directory replacing slashes with dots.

in normal operation mode allows to be configurable through or using specific arguments in the command line, while this is the ideal deployment case, there are scenarios where a more restricted configuration is required: static configuration mode.

Static configuration mode aims to include a built-in configuration in the final binary of Fluent Bit, disabling the usage of external files or flags at runtime.

Getting Started

Requirements

The following steps assumes you are familiar with configuring Fluent Bit using text files and you have experience building it from scratch as described in the section.

Configuration Directory

In your file system prepare a specific directory that will be used as an entry point for the build system to lookup and parse the configuration files. It is mandatory that this directory contain as a minimum one configuration file called fluent-bit.conf containing the required , and sections. As an example create a new fluent-bit.conf file with the following content:

the configuration provided above will calculate CPU metrics from the running system and print them to the standard output interface.

Build with Custom Configuration

Inside Fluent Bit source code, get into the build/ directory and run CMake appending the FLB_STATIC_CONF option pointing the configuration directory recently created, e.g:

then build it:

At this point the fluent-bit binary generated is ready to run without necessity of further configuration:

You can download the most recent stable or development source code.

Stable

For production systems, it's strongly suggested that you get the latest stable release of the source code in either zip file or tarball file format from GitHub using the following link pattern:

For example, for version 1.8.12 the link is:

Development

If you want to contribute to Fluent Bit, you should use the most recent code. You can get the development version from the Git repository:

The master branch is where the development of Fluent Bit happens. Development version users should expect issues when compiling or at run time.

Fluent Bit users are encouraged to help test every development version to ensure a stable release.

Fluent Bit is distributed as fluent-bit package and is available for long-term support releases of Ubuntu. The latest officially supported version is Noble Numbat (24.04).

Single line install

A simple installation script is provided to be used for most Linux targets. This will always install the most recent version released.

This is purely a convenience helper and should always be validated prior to use. The recommended secure deployment approach is to follow the instructions below.

Server GPG key

The first step is to add our server GPG key to your keyring to ensure you can get our signed packages. Follow the official Debian wiki guidance:

Updated key from March 2022

From the 1.9.0 and 1.8.15 releases please note that the GPG key has been updated at so ensure this new one is added.

The GPG Key fingerprint of the new key is:

The previous key is still available at and may be required to install previous versions.

The GPG Key fingerprint of the old key is:

Update your sources lists

Update your repositories database

Now let your system update the apt database:

Install Fluent Bit

Using the following apt-get command you are able now to install the latest fluent-bit:

Now the following step is to instruct systemd to enable the service:

If you do a status check, you should see a similar output like this:

The default configuration of fluent-bit is collecting metrics of CPU usage and sending the records to the standard output, you can see the outgoing data in your /var/log/syslog file.

Fluent Bit is distributed as fluent-bit package and is available for the Raspberry, specifically for distribution, the following versions are supported:

• Raspbian Bullseye (11)

• Raspbian Buster (10)

Server GPG key

The first step is to add our server GPG key to your keyring, on that way you can get our signed packages:

Updated key from March 2022

From the 1.9.0 and 1.8.15 releases please note that the GPG key has been updated at so ensure this new one is added.

The GPG Key fingerprint of the new key is:

The previous key is still available at and may be required to install previous versions.

The GPG Key fingerprint of the old key is:

Refer to the to see which platforms are supported in each release.

Update your sources lists

On Debian and derivative systems such as Raspbian, you need to add our APT server entry to your sources lists, please add the following content at bottom of your /etc/apt/sources.list file.

Raspbian 11 (Bullseye)

Raspbian 10 (Buster)

Update your repositories database

Now let your system update the apt database:

Install Fluent Bit

Using the following apt-get command you are able now to install the latest fluent-bit:

Now the following step is to instruct systemd to enable the service:

If you do a status check, you should see a similar output like this:

The default configuration of fluent-bit is collecting metrics of CPU usage and sending the records to the standard output, you can see the outgoing data in your /var/log/syslog file.

The most secure option is to create the repositories according to the instructions for your specific OS.

A simple installation script is provided to be used for most Linux targets. This will by default install the most recent version released.

curl https://raw.githubusercontent.com/fluent/fluent-bit/master/install.sh | sh

This is purely a convenience helper and should always be validated prior to use.

GPG key updates

From the 1.9.0 and 1.8.15 releases please note that the GPG key has been updated at https://packages.fluentbit.io/fluentbit.key so ensure this new one is added.

The GPG Key fingerprint of the new key is:

C3C0 A285 34B9 293E AF51  FABD 9F9D DC08 3888 C1CD
Fluentbit releases (Releases signing key) <releases@fluentbit.io>

The previous key is still available at https://packages.fluentbit.io/fluentbit-legacy.key and may be required to install previous versions.

The GPG Key fingerprint of the old key is:

F209 D876 2A60 CD49 E680 633B 4FF8 368B 6EA0 722A

Refer to the supported platform documentation to see which platforms are supported in each release.

Migration to Fluent Bit

From version 1.9, td-agent-bit is a deprecated package and is removed after 1.9.9. The correct package name to use now is fluent-bit.

Learn how to install Fluent Bit and the AWS output plugins on Amazon Linux 2 using AWS Systems Manager.

Install on Amazon Linux

Fluent Bit is distributed as fluent-bit package and is available for the latest Amazon Linux 2 and Amazon Linux 2023. The following architectures are supported

• x86_64

• aarch64 / arm64v8

Single line install

A simple installation script is provided to be used for most Linux targets. This will always install the most recent version released.

curl https://raw.githubusercontent.com/fluent/fluent-bit/master/install.sh | sh

This is purely a convenience helper and should always be validated prior to use. The recommended secure deployment approach is to follow the instructions below.

Amazon Linux 2022

Amazon Linux 2022 was previously supported but is removed since it became GA Amazon Linux 2023

Configure Yum

We provide fluent-bit through a Yum repository. In order to add the repository reference to your system, please add a new file called fluent-bit.repo in /etc/yum.repos.d/ with the following content:

Amazon Linux 2

[fluent-bit]
name = Fluent Bit
baseurl = https://packages.fluentbit.io/amazonlinux/2/
gpgcheck=1
gpgkey=https://packages.fluentbit.io/fluentbit.key
enabled=1

Amazon Linux 2023

[fluent-bit]
name = Fluent Bit
baseurl = https://packages.fluentbit.io/amazonlinux/2023/
gpgcheck=1
gpgkey=https://packages.fluentbit.io/fluentbit.key
enabled=1

Note: we encourage you always enable the gpgcheck for security reasons. All our packages are signed.

Updated key from March 2022

From the 1.9.0 and 1.8.15 releases please note that the GPG key has been updated at https://packages.fluentbit.io/fluentbit.key so ensure this new one is added.

The GPG Key fingerprint of the new key is:

C3C0 A285 34B9 293E AF51  FABD 9F9D DC08 3888 C1CD
Fluentbit releases (Releases signing key) <releases@fluentbit.io>

The previous key is still available at https://packages.fluentbit.io/fluentbit-legacy.key and may be required to install previous versions.

The GPG Key fingerprint of the old key is:

F209 D876 2A60 CD49 E680 633B 4FF8 368B 6EA0 722A

Refer to the supported platform documentation to see which platforms are supported in each release.

Install

Once your repository is configured, run the following command to install it:

sudo yum install fluent-bit

Now the following step is to instruct systemd to enable the service:

sudo systemctl start fluent-bit

If you do a status check, you should see a similar output like this:

$ systemctl status fluent-bit
● fluent-bit.service - Fluent Bit
   Loaded: loaded (/usr/lib/systemd/system/fluent-bit.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2016-07-07 02:08:01 BST; 9s ago
 Main PID: 3820 (fluent-bit)
   CGroup: /system.slice/fluent-bit.service
           └─3820 /opt/fluent-bit/bin/fluent-bit -c /etc/fluent-bit/fluent-bit.conf
...

The default configuration of fluent-bit is collecting metrics of CPU usage and sending the records to the standard output, you can see the outgoing data in your /var/log/messages file.

AWS maintains a distribution of Fluent Bit combining the latest official release with a set of Go Plugins for sending logs to AWS services. AWS and Fluent Bit are working together to rewrite their plugins for inclusion in the official Fluent Bit distribution.

Plugins

Currently, the AWS for Fluent Bit image contains Go Plugins for:

• Amazon CloudWatch Logs

• Amazon Kinesis Firehose

• Amazon Kinesis Streams

Fluent Bit includes Amazon CloudWatch Logs plugin named cloudwatch_logs, Amazon Kinesis Firehose plugin named kinesis_firehose and Amazon Kinesis Data Streams plugin named kinesis_streams which are higher performance than Go plugins.

• Amazon CloudWatch

• Amazon Kinesis Data Firehose

• Amazon Kinesis Data Streams

Also, Fluent Bit includes S3 output plugin named s3.

• Amazon S3

Versions and Regional Repositories

AWS vends their container image via Docker Hub, and a set of highly available regional Amazon ECR repositories. For more information, see the AWS for Fluent Bit GitHub repo.

The AWS for Fluent Bit image uses a custom versioning scheme because it contains multiple projects. To see what each release contains, check out the release notes on GitHub.

SSM Public Parameters

AWS vends SSM Public Parameters with the regional repository link for each image. These parameters can be queried by any AWS account.

To see a list of available version tags in a given region, run the following command:

aws ssm get-parameters-by-path --region eu-central-1 --path /aws/service/aws-for-fluent-bit/ --query 'Parameters[*].Name'

To see the ECR repository URI for a given image tag in a given region, run the following:

$ aws ssm get-parameter --region ap-northeast-1 --name /aws/service/aws-for-fluent-bit/2.0.0

You can use these SSM public parameters as parameters in your CloudFormation templates:

Parameters:
  FireLensImage:
    Description: Fluent Bit image for the FireLens Container
    Type: AWS::SSM::Parameter::Value<String>
    Default: /aws/service/aws-for-fluent-bit/latest

Install on Redhat / CentOS

Fluent Bit is distributed as fluent-bit package and is available for the latest stable CentOS system.

The following architectures are supported

• x86_64

• aarch64 / arm64v8

For CentOS 9+ we use CentOS Stream as the canonical base system.

Single line install

A simple installation script is provided to be used for most Linux targets. This will always install the most recent version released.

curl https://raw.githubusercontent.com/fluent/fluent-bit/master/install.sh | sh

This is purely a convenience helper and should always be validated prior to use. The recommended secure deployment approach is to follow the instructions below.

CentOS 8

CentOS 8 is now EOL so the default Yum repositories are unavailable.

Make sure to configure to use an appropriate mirror, for example:

$ sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-* && \
  sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*

An alternative is to use Rocky or Alma Linux which should be equivalent.

Configure Yum

We provide fluent-bit through a Yum repository. In order to add the repository reference to your system, please add a new file called fluent-bit.repo in /etc/yum.repos.d/ with the following content:

[fluent-bit]
name = Fluent Bit
baseurl = https://packages.fluentbit.io/centos/$releasever/
gpgcheck=1
gpgkey=https://packages.fluentbit.io/fluentbit.key
repo_gpgcheck=1
enabled=1

It is best practice to always enable the gpgcheck and repo_gpgcheck for security reasons. We sign our repository metadata as well as all of our packages.

Updated key from March 2022

From the 1.9.0 and 1.8.15 releases please note that the GPG key has been updated at https://packages.fluentbit.io/fluentbit.key so ensure this new one is added.

The GPG Key fingerprint of the new key is:

C3C0 A285 34B9 293E AF51  FABD 9F9D DC08 3888 C1CD
Fluentbit releases (Releases signing key) <releases@fluentbit.io>

The previous key is still available at https://packages.fluentbit.io/fluentbit-legacy.key and may be required to install previous versions.

The GPG Key fingerprint of the old key is:

F209 D876 2A60 CD49 E680 633B 4FF8 368B 6EA0 722A

Refer to the supported platform documentation to see which platforms are supported in each release.

Install

Once your repository is configured, run the following command to install it:

sudo yum install fluent-bit

Now the following step is to instruct Systemd to enable the service:

sudo systemctl start fluent-bit

If you do a status check, you should see a similar output like this:

$ systemctl status fluent-bit
● fluent-bit.service - Fluent Bit
   Loaded: loaded (/usr/lib/systemd/system/fluent-bit.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2016-07-07 02:08:01 BST; 9s ago
 Main PID: 3820 (fluent-bit)
   CGroup: /system.slice/fluent-bit.service
           └─3820 /opt/fluent-bit/bin/fluent-bit -c etc/fluent-bit/fluent-bit.conf
...

The default configuration of fluent-bit is collecting metrics of CPU usage and sending the records to the standard output, you can see the outgoing data in your /var/log/messages file.

FAQ

Yum install fails with a "404 - Page not found" error for the package mirror

The fluent-bit.repo file for the latest installations of Fluent-Bit uses a $releasever variable to determine the correct version of the package to install to your system:

[fluent-bit]
name = Fluent Bit
baseurl = https://packages.fluentbit.io/centos/$releasever/$basearch/
...

Depending on your Red Hat distribution version, this variable may return a value other than the OS major release version (e.g., RHEL7 Server distributions return "7Server" instead of just "7"). The Fluent-Bit package url uses just the major OS release version, so any other value here will cause a 404.

In order to resolve this issue, you can replace the $releasever variable with your system's OS major release version. For example:

[fluent-bit]
name = Fluent Bit
baseurl = https://packages.fluentbit.io/centos/7/$basearch/
gpgcheck=1
gpgkey=https://packages.fluentbit.io/fluentbit.key
repo_gpgcheck=1
enabled=1

Fluent Bit is distributed as fluent-bit package and is available for the latest (and legacy) stable Debian systems: Bookworm and Bullseye. The following architectures are supported

• x86_64

• aarch64 / arm64v8

Single line install

A simple installation script is provided to be used for most Linux targets. This will always install the most recent version released.

curl https://raw.githubusercontent.com/fluent/fluent-bit/master/install.sh | sh

This is purely a convenience helper and should always be validated prior to use. The recommended secure deployment approach is to follow the instructions below.

Server GPG key

The first step is to add our server GPG key to your keyring, on that way you can get our signed packages. Follow the official Debian wiki guidance: https://wiki.debian.org/DebianRepository/UseThirdParty#OpenPGP\_Key\_distribution

curl https://packages.fluentbit.io/fluentbit.key | gpg --dearmor > /usr/share/keyrings/fluentbit-keyring.gpg

Updated key from March 2022

From the 1.9.0 and 1.8.15 releases please note that the GPG key has been updated at https://packages.fluentbit.io/fluentbit.key so ensure this new one is added.

The GPG Key fingerprint of the new key is:

C3C0 A285 34B9 293E AF51  FABD 9F9D DC08 3888 C1CD
Fluentbit releases (Releases signing key) <releases@fluentbit.io>

The previous key is still available at https://packages.fluentbit.io/fluentbit-legacy.key and may be required to install previous versions.

The GPG Key fingerprint of the old key is:

F209 D876 2A60 CD49 E680 633B 4FF8 368B 6EA0 722A

Refer to the supported platform documentation to see which platforms are supported in each release.

Update your sources lists

On Debian, you need to add our APT server entry to your sources lists, please add the following content at bottom of your /etc/apt/sources.list file - ensure to set CODENAME to your specific Debian release name (e.g. bookworm for Debian 12):

deb [signed-by=/usr/share/keyrings/fluentbit-keyring.gpg] https://packages.fluentbit.io/debian/${CODENAME} ${CODENAME} main

Update your repositories database

Now let your system update the apt database:

sudo apt-get update

Install Fluent Bit

Using the following apt-get command you are able now to install the latest fluent-bit:

sudo apt-get install fluent-bit

Now the following step is to instruct systemd to enable the service:

sudo systemctl start fluent-bit

If you do a status check, you should see a similar output like this:

sudo service fluent-bit status
● fluent-bit.service - Fluent Bit
   Loaded: loaded (/lib/systemd/system/fluent-bit.service; disabled; vendor preset: enabled)
   Active: active (running) since mié 2016-07-06 16:58:25 CST; 2h 45min ago
 Main PID: 6739 (fluent-bit)
    Tasks: 1
   Memory: 656.0K
      CPU: 1.393s
   CGroup: /system.slice/fluent-bit.service
           └─6739 /opt/fluent-bit/bin/fluent-bit -c /etc/fluent-bit/fluent-bit.conf
...

The default configuration of fluent-bit is collecting metrics of CPU usage and sending the records to the standard output, you can see the outgoing data in your /var/log/syslog file.

Currently, Fluent Bit supports two configuration formats:

• Yaml: standard configuration format as of v3.2.

• Classic mode: to be deprecated at the end of 2025.

Command line interface

Fluent Bit exposes most of it features through the command line interface. Running the -h option you can get a list of the options available:

$ docker run --rm -it fluent/fluent-bit --help
Usage: /fluent-bit/bin/fluent-bit [OPTION]

Available Options
  -b  --storage_path=PATH specify a storage buffering path
  -c  --config=FILE       specify an optional configuration file
  -d, --daemon            run Fluent Bit in background mode
  -D, --dry-run           dry run
  -f, --flush=SECONDS     flush timeout in seconds (default: 1)
  -C, --custom=CUSTOM     enable a custom plugin
  -i, --input=INPUT       set an input
  -F  --filter=FILTER     set a filter
  -m, --match=MATCH       set plugin match, same as '-p match=abc'
  -o, --output=OUTPUT     set an output
  -p, --prop="A=B"        set plugin configuration property
  -R, --parser=FILE       specify a parser configuration file
  -e, --plugin=FILE       load an external plugin (shared lib)
  -l, --log_file=FILE     write log info to a file
  -t, --tag=TAG           set plugin tag, same as '-p tag=abc'
  -T, --sp-task=SQL       define a stream processor task
  -v, --verbose           increase logging verbosity (default: info)
  -w, --workdir           set the working directory
  -H, --http              enable monitoring HTTP server
  -P, --port              set HTTP server TCP port (default: 2020)
  -s, --coro_stack_size   set coroutines stack size in bytes (default: 24576)
  -q, --quiet             quiet mode
  -S, --sosreport         support report for Enterprise customers
  -V, --version           show version number
  -h, --help              print this help

Inputs
  cpu                     CPU Usage
  mem                     Memory Usage
  thermal                 Thermal
  kmsg                    Kernel Log Buffer
  proc                    Check Process health
  disk                    Diskstats
  systemd                 Systemd (Journal) reader
  netif                   Network Interface Usage
  docker                  Docker containers metrics
  docker_events           Docker events
  node_exporter_metrics   Node Exporter Metrics (Prometheus Compatible)
  fluentbit_metrics       Fluent Bit internal metrics
  prometheus_scrape       Scrape metrics from Prometheus Endpoint
  tail                    Tail files
  dummy                   Generate dummy data
  dummy_thread            Generate dummy data in a separate thread
  head                    Head Input
  health                  Check TCP server health
  http                    HTTP
  collectd                collectd input plugin
  statsd                  StatsD input plugin
  opentelemetry           OpenTelemetry
  nginx_metrics           Nginx status metrics
  serial                  Serial input
  stdin                   Standard Input
  syslog                  Syslog
  tcp                     TCP
  mqtt                    MQTT, listen for Publish messages
  forward                 Fluentd in-forward
  random                  Random

Filters
  alter_size              Alter incoming chunk size
  aws                     Add AWS Metadata
  checklist               Check records and flag them
  record_modifier         modify record
  throttle                Throttle messages using sliding window algorithm
  type_converter          Data type converter
  kubernetes              Filter to append Kubernetes metadata
  modify                  modify records by applying rules
  multiline               Concatenate multiline messages
  nest                    nest events by specified field values
  parser                  Parse events
  expect                  Validate expected keys and values
  grep                    grep events by specified field values
  rewrite_tag             Rewrite records tags
  lua                     Lua Scripting Filter
  stdout                  Filter events to STDOUT
  geoip2                  add geoip information to records
  nightfall               scans records for sensitive content

Outputs
  azure                   Send events to Azure HTTP Event Collector
  azure_blob              Azure Blob Storage
  azure_kusto             Send events to Kusto (Azure Data Explorer)
  bigquery                Send events to BigQuery via streaming insert
  counter                 Records counter
  datadog                 Send events to DataDog HTTP Event Collector
  es                      Elasticsearch
  exit                    Exit after a number of flushes (test purposes)
  file                    Generate log file
  forward                 Forward (Fluentd protocol)
  http                    HTTP Output
  influxdb                InfluxDB Time Series
  logdna                  LogDNA
  loki                    Loki
  kafka                   Kafka
  kafka-rest              Kafka REST Proxy
  nats                    NATS Server
  nrlogs                  New Relic
  null                    Throws away events
  opensearch              OpenSearch
  plot                    Generate data file for GNU Plot
  pgsql                   PostgreSQL
  skywalking              Send logs into log collector on SkyWalking OAP
  slack                   Send events to a Slack channel
  splunk                  Send events to Splunk HTTP Event Collector
  stackdriver             Send events to Google Stackdriver Logging
  stdout                  Prints events to STDOUT
  syslog                  Syslog
  tcp                     TCP Output
  td                      Treasure Data
  flowcounter             FlowCounter
  gelf                    GELF Output
  websocket               Websocket
  cloudwatch_logs         Send logs to Amazon CloudWatch
  kinesis_firehose        Send logs to Amazon Kinesis Firehose
  kinesis_streams         Send logs to Amazon Kinesis Streams
  opentelemetry           OpenTelemetry
  prometheus_exporter     Prometheus Exporter
  prometheus_remote_write Prometheus remote write
  s3                      Send to S3

Parsers enable Fluent Bit components to transform unstructured data into a structured internal representation. You can define parsers either directly in the main configuration file or in separate external files for better organization.

This page provides a general overview of how to declare parsers.

The main section name is parsers, and it allows you to define a list of parser configurations. The following example demonstrates how to set up two simple parsers:

parsers:
  - name: json
    format: json

  - name: docker
    format: json
    time_key: time
    time_format: "%Y-%m-%dT%H:%M:%S.%L"
    time_keep: true

You can define multiple parsers sections, either within the main configuration file or distributed across included files.

For more detailed information on parser options and advanced configurations, please refer to the Configuring Parsers section.

While Fluent Bit comes with a variety of built-in plugins, it also supports loading external plugins at runtime. This feature is especially useful for loading Go or Wasm plugins that are built as shared object files (.so). Fluent Bit's YAML configuration provides two ways to load these external plugins:

1. Inline YAML Section

You can specify external plugins directly within your main YAML configuration file using the plugins section. Here’s an example:

plugins:
  - /path/to/out_gstdout.so

service:
  log_level: info

pipeline:
  inputs:
    - name: random

  outputs:
    - name: gstdout
      match: '*'

2. YAML Plugins File Included via plugins_file Option

Alternatively, you can load external plugins from a separate YAML file by specifying the plugins_file option in the service section. Here’s how to configure this:

service:
  log_level: info
  plugins_file: extra_plugins.yaml

pipeline:
  inputs:
    - name: random

  outputs:
    - name: gstdout
      match: '*'

In this setup, the extra_plugins.yaml file might contain the following plugins section:

plugins:
  - /other/path/to/out_gstdout.so

Key Points

• Built-in vs. External: Fluent Bit comes with many built-in plugins, but you can load external plugins at runtime to extend the tool’s functionality.

• Loading Mechanism: External plugins must be shared object files (.so). You can define them inline in the main YAML configuration or include them from a separate YAML file for better modularity.

Multiline parsers are used to combine logs that span multiple events into a single, cohesive message. This is particularly useful for handling stack traces, error logs, or any log entry that contains multiple lines of information.

In YAML configuration, the syntax for defining multiline parsers differs slightly from the classic configuration format introducing minor breaking changes, specifically on how the rules are defined.

Below is an example demonstrating how to define a multiline parser directly in the main configuration file, as well as how to include additional definitions from external files:

multiline_parsers:
  - name: multiline-regex-test
    type: regex
    flush_timeout: 1000
    rules:
      - state: start_state
        regex: '/([a-zA-Z]+ \d+ \d+:\d+:\d+)(.*)/'
        next_state: cont
      - state: cont
        regex: '/^\s+at.*/'
        next_state: cont

The example above defines a multiline parser named multiline-regex-test that uses regular expressions to handle multi-event logs. The parser contains two rules: the first rule transitions from start_state to cont when a matching log entry is detected, and the second rule continues to match subsequent lines.

For more detailed information on configuring multiline parsers, including advanced options and use cases, please refer to the Configuring Multiline Parsers section.

The includes section allows you to specify additional YAML configuration files to be merged into the current configuration. These files are identified as a list of filenames and can include relative or absolute paths. If no absolute path is provided, the file is assumed to be located in a directory relative to the file that references it.

This feature is useful for organizing complex configurations into smaller, manageable files and including them as needed.

Usage

Below is an example demonstrating how to include additional YAML files using relative path references. This is the file system path structure

├── fluent-bit.yaml
├── inclusion-1.yaml
└── subdir
    └── inclusion-2.yaml

The content of fluent-bit.yaml

includes:
  - inclusion-1.yaml
  - subdir/inclusion-2.yaml

Key Points

• Relative Paths: If a path is not specified as absolute, it will be treated as relative to the file that includes it.

• Organized Configurations: Using the includes section helps keep your configuration modular and easier to maintain.

note: Ensure that the included files are formatted correctly and contain valid YAML configurations for seamless integration.

The Upstream Servers section defines a group of endpoints, referred to as nodes, which are used by output plugins to distribute data in a round-robin fashion. This is particularly useful for plugins that require load balancing when sending data. Examples of plugins that support this capability include Forward and Elasticsearch.

In YAML, this section is named upstream_servers and requires specifying a name for the group and a list of nodes. Below is an example that defines two upstream server groups: forward-balancing and forward-balancing-2:

upstream_servers:
  - name: forward-balancing
    nodes:
      - name: node-1
        host: 127.0.0.1
        port: 43000

      - name: node-2
        host: 127.0.0.1
        port: 44000

      - name: node-3
        host: 127.0.0.1
        port: 45000
        tls: true
        tls_verify: false
        shared_key: secret

  - name: forward-balancing-2
    nodes:
      - name: node-A
        host: 192.168.1.10
        port: 50000

      - name: node-B
        host: 192.168.1.11
        port: 51000

Key Concepts

• Nodes: Each node in the upstream_servers group must specify a name, host, and port. Additional settings like tls, tls_verify, and shared_key can be configured as needed for secure communication.

Usage Note

While the upstream_servers section can be defined globally, some output plugins may require the configuration to be specified in a separate YAML file. Be sure to consult the documentation for each specific output plugin to understand its requirements.

For more details, refer to the documentation of the respective output plugins.

Install Fluent Bit in your embedded Linux system.

Install

To install, select Fluent Bit in your defconfig. See the Config.in file for all configuration options.

BR2_PACKAGE_FLUENT_BIT=y

Run

The default configuration file is written to:

/etc/fluent-bit/fluent-bit.conf

Fluent Bit is started by the S99fluent-bit script.

Support

All configurations with a toolchain that supports threads and dynamic library linking are supported.

Fluent Bit is compatible with latest Apple macOS system on x86_64 and Apple Silicon architectures.

Installation Packages

The packages can be found here: https://packages.fluentbit.io/macos/

Requirements

For the next steps, you will need to have Homebrew installed in your system. If is not there, you can install it with the following command:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Installing from Homebrew

The Fluent Bit package on Homebrew is not officially supported, but should work for basic use cases and testing. It can be installed using:

brew install fluent-bit

Compile from Source

Install build dependencies

Run the following brew command in your terminal to retrieve the dependencies:

brew install git cmake openssl bison

Get the source and build it

Grab a fresh copy of the Fluent Bit source code (upstream):

git clone https://github.com/fluent/fluent-bit
cd fluent-bit

Optionally, if you want to use a specific version, just checkout to the proper tag. If you want to use v1.8.13 just do:

git checkout v1.8.13

In order to prepare the build system, we need to expose certain environment variables so Fluent Bit CMake build rules can pick the right libraries:

export OPENSSL_ROOT_DIR=`brew --prefix openssl`
export PATH=`brew --prefix bison`/bin:$PATH

Change to the build/ directory inside the Fluent Bit sources:

cd build/

Build Fluent Bit. Note that we are indicating to the build system "where" the final binaries and config files should be installed:

cmake -DFLB_DEV=on -DCMAKE_INSTALL_PREFIX=/opt/fluent-bit ../
make -j 16

Install Fluent Bit to the directory specified above. Note that this requires root privileges due to the directory we will write information to:

sudo make install

The binaries and configuration examples can be located at /opt/fluent-bit/.

Create macOS installer from source

Grab a fresh copy of the Fluent Bit source code (upstream):

git clone https://github.com/fluent/fluent-bit
cd fluent-bit

Optionally, if you want to use a specific version, just checkout to the proper tag. If you want to use v1.9.2 just do:

git checkout v1.9.2

In order to prepare the build system, we need to expose certain environment variables so Fluent Bit CMake build rules can pick the right libraries:

export OPENSSL_ROOT_DIR=`brew --prefix openssl`
export PATH=`brew --prefix bison`/bin:$PATH

And then, creating the specific macOS SDK target (For example, specifying macOS Big Sur (11.3) SDK environment):

export MACOSX_DEPLOYMENT_TARGET=11.3

Change to the build/ directory inside the Fluent Bit sources:

cd build/

Build the Fluent Bit macOS installer.

cmake -DCPACK_GENERATOR=productbuild -DCMAKE_INSTALL_PREFIX=/opt/fluent-bit ../
make -j 16
cpack -G productbuild

Then, macOS installer will be generated as:

CPack: Create package using productbuild
CPack: Install projects
CPack: - Run preinstall target for: fluent-bit
CPack: - Install project: fluent-bit []
CPack: -   Install component: binary
CPack: -   Install component: library
CPack: -   Install component: headers
CPack: -   Install component: headers-extra
CPack: Create package
CPack: -   Building component package: /Users/fluent-bit-builder/GitHub/fluent-bit/build/_CPack_Packages/Darwin/productbuild//Users/fluent-bit-builder/GitHub/fluent-bit/build/fluent-bit-1.9.2-apple/Contents/Packages/fluent-bit-1.9.2-apple-binary.pkg
CPack: -   Building component package: /Users/fluent-bit-builder/GitHub/fluent-bit/build/_CPack_Packages/Darwin/productbuild//Users/fluent-bit-builder/GitHub/fluent-bit/build/fluent-bit-1.9.2-apple/Contents/Packages/fluent-bit-1.9.2-apple-headers.pkg
CPack: -   Building component package: /Users/fluent-bit-builder/GitHub/fluent-bit/build/_CPack_Packages/Darwin/productbuild//Users/fluent-bit-builder/GitHub/fluent-bit/build/fluent-bit-1.9.2-apple/Contents/Packages/fluent-bit-1.9.2-apple-headers-extra.pkg
CPack: -   Building component package: /Users/fluent-bit-builder/GitHub/fluent-bit/build/_CPack_Packages/Darwin/productbuild//Users/fluent-bit-builder/GitHub/fluent-bit/build/fluent-bit-1.9.2-apple/Contents/Packages/fluent-bit-1.9.2-apple-library.pkg
CPack: - package: /Users/fluent-bit-builder/GitHub/fluent-bit/build/fluent-bit-1.9.2-apple.pkg generated.

Finally, fluent-bit-<fluent-bit version>-(intel or apple).pkg will be generated.

The created installer will put binaries at /opt/fluent-bit/.

Running Fluent Bit

To make the access path easier to Fluent Bit binary, in your terminal extend the PATH variable:

export PATH=/opt/fluent-bit/bin:$PATH

Now as a simple test, try Fluent Bit by generating a simple dummy message which will be printed to the standard output interface every 1 second:

 fluent-bit -i dummy -o stdout -f 1

You will see an output similar to this:

Fluent Bit v1.9.0
* Copyright (C) 2015-2021 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2022/02/08 17:13:52] [ info] [engine] started (pid=14160)
[2022/02/08 17:13:52] [ info] [storage] version=1.1.6, initializing...
[2022/02/08 17:13:52] [ info] [storage] in-memory
[2022/02/08 17:13:52] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2022/02/08 17:13:52] [ info] [cmetrics] version=0.2.2
[2022/02/08 17:13:52] [ info] [sp] stream processor started
[0] dummy.0: [1644362033.676766000, {"message"=>"dummy"}]
[0] dummy.0: [1644362034.676914000, {"message"=>"dummy"}]

To halt the process, press ctrl-c in the terminal.

Fluent Bit might optionally use a configuration file to define how the service will behave.

Before proceeding we need to understand how the configuration schema works.

The schema is defined by three concepts:

• Sections

• Entries: Key/Value

• Indented Configuration Mode

A simple example of a configuration file is as follows:

[SERVICE]
    # This is a commented line
    Daemon    off
    log_level debug

Sections

A section is defined by a name or title inside brackets. Looking at the example above, a Service section has been set using [SERVICE] definition. Section rules:

• All section content must be indented (4 spaces ideally).

• Multiple sections can exist on the same file.

• A section is expected to have comments and entries, it cannot be empty.

• Any commented line under a section, must be indented too.

• End-of-line comments are not supported, only full-line comments.

Entries: Key/Value

A section may contain Entries, an entry is defined by a line of text that contains a Key and a Value, using the above example, the [SERVICE] section contains two entries, one is the key Daemon with value off and the other is the key Log_Level with the value debug. Entries rules:

• An entry is defined by a key and a value.

• A key must be indented.

• A key must contain a value which ends in the breakline.

• Multiple keys with the same name can exist.

Also commented lines are set prefixing the # character, those lines are not processed but they must be indented too.

Indented Configuration Mode

Fluent Bit configuration files are based in a strict Indented Mode, that means that each configuration file must follow the same pattern of alignment from left to right when writing text. By default an indentation level of four spaces from left to right is suggested. Example:

[FIRST_SECTION]
    # This is a commented line
    Key1  some value
    Key2  another value
    # more comments

[SECOND_SECTION]
    KeyN  3.14

As you can see there are two sections with multiple entries and comments, note also that empty lines are allowed and they do not need to be indented.