# Fluent Bit documentation
High Performance Telemetry Agent for Logs, Metrics and Traces
[Fluent Bit](https://fluentbit.io) is a fast and lightweight telemetry agent for logs, metrics, and traces for Linux, macOS, Windows, and BSD family operating systems. Fluent Bit has been made with a strong focus on performance to allow the collection and processing of telemetry data from different sources without complexity.
## Features
* High performance: High throughput with low resources consumption
* Data parsing
* Convert your unstructured messages using Fluent Bit parsers: [JSON](/manual/data-pipeline/parsers/json), [Regex](/manual/data-pipeline/parsers/regular-expression), [LTSV](/manual/data-pipeline/parsers/ltsv) and [Logfmt](/manual/data-pipeline/parsers/logfmt)
* Metrics support: Prometheus and OpenTelemetry compatible
* Reliability and data integrity
* [Backpressure](/manual/administration/backpressure) handling
* [Data buffering](/manual/data-pipeline/buffering) in memory and file system
* Networking
* Security: Built-in TLS/SSL support
* Asynchronous I/O
* Pluggable architecture and [extensibility](/manual/fluent-bit-for-developers/library_api): Inputs, Filters and Outputs:
* Connect nearly any source to nearly any destination using preexisting plugins
* Extensibility:
* Write input, filter, or output plugins in the C language
* Wasm: [Wasm Filter Plugins](/manual/fluent-bit-for-developers/wasm-filter-plugins) or [Wasm Input Plugins](/manual/fluent-bit-for-developers/wasm-input-plugins)
* Write [Filters in Lua](/manual/data-pipeline/filters/lua) or [Output plugins in Golang](/manual/fluent-bit-for-developers/golang-output-plugins)
* [Monitoring](/manual/administration/monitoring): Expose internal metrics over HTTP in JSON and [Prometheus](https://prometheus.io/) format
* [Stream Processing](/manual/stream-processing/overview): Perform data selection and transformation using basic SQL queries
* Create new streams of data using query results
* Aggregation windows
* Data analysis and prediction: Time series forecasting
* Portable: Runs on Linux, macOS, Windows and BSD systems
### Release notes
For more details about changes in each release, refer to the [official release notes](https://fluentbit.io/announcements/).
If you are upgrading from the Fluent Bit `4.2` series, start with [What's new in Fluent Bit v5.0](/manual/installation/whats-new-in-fluent-bit-v5.0) and [Upgrade notes](/manual/installation/upgrade-notes).
## Fluent Bit, Fluentd, and CNCF
Fluent Bit is a [CNCF](https://www.cncf.io/) graduated sub-project under the umbrella of [Fluentd](https://www.fluentd.org).
Fluent Bit was originally created by [Eduardo Silva](https://www.linkedin.com/in/edsiper/) and is now sponsored by [Chronosphere](https://chronosphere.io/). As a CNCF-hosted project, it's a fully vendor-neutral and community-driven project.
## License
Fluent Bit, including its core, plugins, and tools, is distributed under the terms of the [Apache License v2.0](https://github.com/fluent/fluent-bit?tab=Apache-2.0-1-ov-file#readme).
# What's Fluent Bit?
Fluent Bit is a CNCF graduated project under the umbrella of Fluentd
[Fluent Bit](https://fluentbit.io) is an open source telemetry agent that processes logs, metrics, traces, and profiles. It's designed to efficiently handle the challenges of collecting and processing telemetry data across a wide range of environments, from constrained systems to complex cloud infrastructures. Managing telemetry data from various sources and formats can be a constant challenge, particularly when performance is a critical factor.
Rather than serving as a drop-in replacement, Fluent Bit enhances the observability strategy for your infrastructure. It adapts and optimizes your existing logging layer, and adds metrics and traces processing. Fluent Bit supports a vendor-neutral approach, with native OpenTelemetry (OTLP) ingestion and delivery and seamless integration with ecosystems such as Prometheus. Trusted by major cloud providers, banks, and companies that need a ready-to-use telemetry agent, Fluent Bit effectively manages diverse data sources and formats. It maintains optimal performance while keeping resource consumption low.
Fluent Bit can be deployed as an edge agent for localized telemetry data handling or utilized as a central aggregator/collector for managing telemetry data across multiple sources and environments.
{% embed url="" %}
## The history of Fluent Bit
In 2014, the [Fluentd](https://www.fluentd.org/) team at [Treasure Data](https://www.treasure.ai/) was forecasting the need for a lightweight log processor for constraint environments like embedded Linux and gateways. To meet this need, Eduardo Silva created Fluent Bit, a new open source solution and part of the Fluentd ecosystem.
After the project matured, it gained traction for normal Linux systems. With the new containerized world, the cloud native community asked to extend the project scope to support more sources, filters, and destinations. Not long after, Fluent Bit became one of the preferred solutions to solve the logging challenges in cloud environments.
# Fluentd and Fluent Bit
The production grade telemetry ecosystem
Telemetry data processing can be complex, especially at scale. That's why [Fluentd](https://www.fluentd.org) was created. Fluentd is more than a basic tool. It's grown into a full-scale ecosystem that contains SDKs for different languages and sub-projects, like [Fluent Bit](https://fluentbit.io).
The Fluentd and Fluent Bit projects are both:
* Licensed under the terms of Apache License v2.0.
* Graduated hosted projects by the [Cloud Native Computing Foundation (CNCF)](https://www.cncf.io/).
* Production-grade solutions, with Fluent Bit deployed over 15 billion times globally.
* Vendor neutral and community driven.
* Widely adopted by the industry, being trusted by major companies like Amazon, Microsoft, Google, and hundreds of others.
The projects have many similarities: [Fluent Bit](https://fluentbit.io) is designed and built on top of the best ideas of [Fluentd](https://www.fluentd.org) architecture and general design. Which one you choose depends on your end-users' needs.
The following table describes a comparison of different areas of the projects:
| Attribute | Fluentd | Fluent Bit |
| ------------- | -------------------------------------------------------------- | -------------------------------------------------------------- |
| Scope | Containers / Servers | Embedded Linux / Containers / Servers |
| Language | C and Ruby | C |
| Memory | Greater than 60 MB | Approximately 450 KB |
| Performance | Medium Performance | High Performance |
| Dependencies | Built as a Ruby Gem, depends on other gems. | Zero dependencies, unless required by a plugin. |
| Plugins | Over 1,000 external plugins available. | Over 100 built-in plugins available. |
| OpenTelemetry | Available through plugins. | Native OTLP ingestion and delivery. |
| License | [Apache License v2.0](https://apache.org/licenses/LICENSE-2.0) | [Apache License v2.0](https://apache.org/licenses/LICENSE-2.0) |
Both [Fluentd](https://www.fluentd.org) and [Fluent Bit](https://fluentbit.io) can work as Aggregators or Forwarders, and can complement each other or be used as standalone solutions.
In the recent years, cloud providers have switched from Fluentd to Fluent Bit for performance and compatibility. Fluent Bit is now considered the next-generation solution.
# Lab resources
An overview of free public labs for learning how to successfully use Fluent Bit.
To use these lab resources to their full extent, you'll need to [install](/manual/installation/downloads) resources in your local environment to test and run Fluent Bit.
## O11y workshops by Chronosphere
Chronosphere provides several [open source observability workshops](https://o11y-workshops.gitlab.io/), including a Fluent Bit workshop:
{% embed url="" %}
Fluent Bit workshop for getting started with cloud native telemetry pipelines
{% endembed %}
1. [Lab 1 - Introduction to Fluent Bit](https://o11y-workshops.gitlab.io/workshop-fluentbit/#/3)
2. [Lab 2 - Installing Fluent Bit](https://o11y-workshops.gitlab.io/workshop-fluentbit/#/4)
3. [Lab 3 - Exploring First Pipelines](https://o11y-workshops.gitlab.io/workshop-fluentbit/#/5)
4. [Lab 4 - Exploring More Pipelines](https://o11y-workshops.gitlab.io/workshop-fluentbit/#/6)
5. [Lab 5 - Understanding Backpressure](https://o11y-workshops.gitlab.io/workshop-fluentbit/#/7)
6. [Lab 6 - Avoid Telemetry Data Loss](https://o11y-workshops.gitlab.io/workshop-fluentbit/#/8)
7. [Lab 7 - Pipeline Integration with OpenTelemetry](https://o11y-workshops.gitlab.io/workshop-fluentbit/#/9)
8. [Lab 8 - Controlling logs with Fluent Bit on Kubernetes](https://o11y-workshops.gitlab.io/workshop-fluentbit/#/10)
9. [Lab 9 - Visualizing Fluent Bit telemetry data with OpenSearch](https://o11y-workshops.gitlab.io/workshop-fluentbit/#/11)
10. [Lab 10 - Advanced telemetry routing](https://o11y-workshops.gitlab.io/workshop-fluentbit/#/12)
You can also view the source files for these workshops on [GitLab](https://gitlab.com/o11y-workshops/workshop-fluentbit).
## Fluent Bit pod logging on Kubernetes workshop by Amazon
This workshop by Amazon covers deploying Fluent Bit for pod-level logging on Kubernetes and routing data to CloudWatch Logs.
{% embed url="" %}
# Key concepts
Learn these key concepts to understand how Fluent Bit operates.
Before diving into [Fluent Bit](https://fluentbit.io) you might want to get acquainted with some of the key concepts of the service. This document provides an introduction to those concepts and common [Fluent Bit](https://fluentbit.io) terminology. Reading this document will help you gain a more general understanding of the following topics:
* Event or Record
* Filtering
* Processor
* Tag
* Timestamp
* Match
* Structured Message
## Events or records
Every incoming piece of data that belongs to a log, metric, trace, or profile that's retrieved by Fluent Bit is considered an *Event* or a *Record*.
As an example, consider the following content of a Syslog file:
```
Jan 18 12:52:16 flb systemd[2222]: Starting GNOME Terminal Server
Jan 18 12:52:16 flb dbus-daemon[2243]: [session uid=1000 pid=2243] Successfully activated service 'org.gnome.Terminal'
Jan 18 12:52:16 flb systemd[2222]: Started GNOME Terminal Server.
Jan 18 12:52:16 flb gsd-media-keys[2640]: # watch_fast: "/org/gnome/terminal/legacy/" (establishing: 0, active: 0)
```
It contains four lines that represent four independent Events.
An Event is comprised of:
* timestamp
* key/value metadata (v2.1.0 and greater)
* payload
### Event format
The Fluent Bit wire protocol represents an Event as a two-element array with a nested array as the first element:
```javascript
[[TIMESTAMP, METADATA], MESSAGE]
```
where
* *`TIMESTAMP`* is a timestamp in seconds as an integer or floating point value (not a string).
* *`METADATA`* is an object containing event metadata, and might be empty.
* *`MESSAGE`* is an object containing the event body.
Fluent Bit versions prior to v2.1.0 used:
```javascript
[TIMESTAMP, MESSAGE]
```
to represent events. This format is still supported for reading input event streams.
## Filtering
You might need to perform modifications on an event's content. The process to alter, append to, or drop Events is called [*filtering*](/manual/data-pipeline/filters).
Use filtering to:
* Append specific information to the Event like an IP address or metadata.
* Select a specific piece of the Event content.
* Drop Events that match a certain pattern.
## Processor
[Processors](/manual/data-pipeline/processors) modify, transform, or enhance data as it moves through the pipeline. Unlike filters, processors are attached directly to individual input or output plugins and don't use tag matching. Each processor operates only on data from its associated plugin.
Processors run in the same thread as their associated plugin, which improves throughput compared to filters—especially when [multithreading](/manual/administration/multithreading) is enabled.
Processors are supported in [YAML configuration files](/manual/administration/configuring-fluent-bit/yaml) only, and can act on logs, metrics, traces, and profiles.
## Tag
Every Event ingested by Fluent Bit is assigned a Tag. This tag is an internal string used in a later stage by the Router to decide which Filter or [Output](/manual/data-pipeline/outputs) phase it must go through.
Most tags are assigned manually in the configuration. If a tag isn't specified, Fluent Bit assigns the name of the [Input](/manual/data-pipeline/inputs) plugin instance where that Event was generated from.
{% hint style="info" %}
The [Forward](/manual/data-pipeline/inputs/forward) input plugin doesn't assign tags. This plugin speaks the Fluentd wire protocol called Forward where every Event already comes with a Tag associated. Fluent Bit will always use the incoming Tag set by the client.
{% endhint %}
A tagged record must always have a Matching rule. To learn more about Tags and Matches, see [Routing](/manual/data-pipeline/router).
## Timestamp
The timestamp represents the time an Event was created. Every Event contains an associated timestamp. All events have timestamps, and they're set by the input plugin or discovered through a data parsing process.
The timestamp is a numeric fractional integer in the format:
```javascript
SECONDS.NANOSECONDS
```
where:
* `_SECONDS_` is the number of seconds that have elapsed since the Unix epoch.
* `_NANOSECONDS_` is a fractional second or one thousand-millionth of a second.
## Match
Fluent Bit lets you route your collected and processed Events to one or multiple destinations. A *Match* represents a rule to select Events where a Tag matches a defined rule.
To learn more about Tags and Matches, see [Routing](/manual/data-pipeline/router).
## Structured messages
Source events can have a structure. A structure defines a set of `keys` and `values` inside the Event message to implement faster operations on data modifications. Fluent Bit treats every Event message as a structured message.
Consider the following two messages:
* No structured message
```javascript
"Project Fluent Bit created on 1398289291"
```
* With a structured message
```javascript
{"project": "Fluent Bit", "created": 1398289291}
```
For performance reasons, Fluent Bit uses a binary serialization data format called [MessagePack](https://msgpack.org/).
# Data pipeline
The Fluent Bit data pipeline incorporates several specific concepts. Data processing flows through the pipeline following these concepts in order.
{% @mermaid/diagram content="graph LR
accTitle: Fluent Bit data pipeline
accDescr: A diagram of the Fluent Bit data pipeline, which includes input, processors, a parser, a filter, a buffer, routing, and various outputs.
A\[Input] --> B\[Processors]
B --> C\[Parser]
C --> D\[Filter]
D --> E\[Buffer]
E --> F((Routing))
F --> G\[Output 1]
F --> H\[Output 2]
F --> I\[Output 3]" %}
## Inputs
[Input plugins](/manual/data-pipeline/inputs) gather information from different sources. Some plugins collect data from log files, and others gather metrics information from the operating system. There are many plugins to suit different needs.
## Processors
[Processors](/manual/data-pipeline/processors) are components that modify, transform, or enhance data as it flows through the pipeline. Processors are attached directly to individual input or output plugins rather than defined globally, and they don't use tag matching.
Because processors run in the same thread as their associated plugin, they can reduce performance overhead compared to filters—especially when [multithreading](/manual/administration/multithreading) is enabled.
Processors are configured in [YAML configuration files](/manual/administration/configuring-fluent-bit/yaml) only.
## Parser
[Parsers](/manual/data-pipeline/parsers) convert unstructured data to structured data. Use a parser to set a structure to the incoming data by using input plugins as data is collected.
## Filter
[Filters](/manual/data-pipeline/filters) let you alter the collected data before delivering it to a destination. In production environments you need full control of the data you're collecting. Using filters lets you control data before processing.
## Buffer
The [buffering](/manual/data-pipeline/buffering) phase in the pipeline aims to provide a unified and persistent mechanism to store your data, using the primary in-memory model or the file system-based mode.
## Routing
[Routing](/manual/data-pipeline/router) is a core feature that lets you route your data through filters, and then to one or multiple destinations. The router relies on the concept of [tags](/manual/concepts/key-concepts#tag) and [matching](/manual/concepts/key-concepts#match) rules.
## Output
[Output plugins](/manual/data-pipeline/outputs) let you define destinations for your data. Common destinations are remote services, local file systems, or other standard interfaces.
# Download and install Fluent Bit
Fluent Bit is compatible with most x86-based, x86\_64-based, arm32v7-based, and arm64v8-based systems.
## Build from source code
You can [build and install Fluent Bit from its source code](/manual/installation/downloads/source). There are also platform-specific guides for building Fluent Bit from source on [macOS](/manual/installation/downloads/macos#compile-from-source) and [Windows](/manual/installation/downloads/windows#compile-from-source).
## Supported platforms and packages
To install Fluent Bit from one of the available packages, use the installation method for your chosen platform.
### Container deployment
Fluent Bit is available for the following container deployments:
* [Containers on AWS](/manual/installation/downloads/aws-container)
* [Docker](/manual/installation/downloads/docker)
* [Kubernetes](/manual/installation/downloads/kubernetes)
### Linux
Fluent Bit is available on [Linux](/manual/installation/downloads/linux), including the following distributions:
* [Amazon Linux](/manual/installation/downloads/linux/amazon-linux)
* [Buildroot embedded Linux](/manual/installation/downloads/linux/buildroot-embedded-linux)
* [Debian](/manual/installation/downloads/linux/debian)
* [Raspbian and Raspberry Pi](/manual/installation/downloads/linux/raspbian-raspberry-pi)
* [Red Hat and CentOS](/manual/installation/downloads/linux/redhat-centos)
* [Rocky Linux and Alma Linux](/manual/installation/downloads/linux/alma-rocky)
* [Ubuntu](/manual/installation/downloads/linux/ubuntu)
* [Yocto embedded Linux](/manual/installation/downloads/linux/yocto-embedded-linux)
### macOS
Fluent Bit is available on [macOS](/manual/installation/downloads/macos).
### Windows
Fluent Bit is available on [Windows](/manual/installation/downloads/windows).
### Other platforms
Official support is based on community demand. Fluent Bit might run on older operating systems, but must be built from source or using custom packages.
Fluent Bit can run on Berkeley Software Distribution (BSD) systems and IBM Z Linux (s390x) systems with restrictions. Not all plugins and filters are supported.
## Enterprise providers
Fluent Bit packages are also provided by [enterprise providers](https://fluentbit.io/enterprise/) for older end-of-life versions, Unix systems, or for additional support and features including aspects (such as CVE backporting).
# Build from source code
You can download the most recent stable or development source code.
## Stable
For production systems, it's strongly suggested that you get the latest stable release of the source code in either zip file or tarball file format from GitHub using the following link pattern:
```
https://github.com/fluent/fluent-bit/archive/refs/tags/v<release version>.tar.gz
https://github.com/fluent/fluent-bit/archive/refs/tags/v<release version>.zip
```
For example, for version 1.8.12 the link is:
## Development
If you want to contribute to Fluent Bit, you should use the most recent code. You can get the development version from the Git repository:
```shell
git clone https://github.com/fluent/fluent-bit
```
The `master` branch is where the development of Fluent Bit happens. Development version users should expect issues when compiling or at run time.
Fluent Bit users are encouraged to help test every development version to ensure a stable release.
## Next step
After downloading Fluent Bit, install it using one of the following methods:
* [Build and install](/manual/installation/downloads/source/build-and-install)
* [Build with a static configuration](/manual/installation/downloads/source/build-with-static-configuration)
# Build and install
[Fluent Bit](https://fluentbit.io) uses [CMake](https://cmake.org) as its build system.
## Requirements
To build and install Fluent Bit from source, you must also install the following packages:
* `bison`
* `build-essentials`
* `cmake` (version 3.31.6 or later)
* `flex`
* `libssl-dev`
* `libyaml-dev`
* `pkg-config`
Additionally, certain [input](/manual/data-pipeline/inputs) or [output](/manual/data-pipeline/outputs) plugins might depend on additional components. For example, some plugins require Kafka.
## Prepare environment
If you already know how CMake works, you can skip this section and review the available [build options](#general-options).
The following steps explain how to build and install the project with the default options.
1. Change to the `build/` directory inside the Fluent Bit sources:
```shell
cd build/
```
2. Let [CMake](https://cmake.org) configure the project specifying where the root path is located:
```shell
cmake ../
```
This command displays a series of results similar to:
```
-- The C compiler identification is GNU 4.9.2
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- The CXX compiler identification is GNU 4.9.2
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
...
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Looking for accept4
-- Looking for accept4 - not found
-- Configuring done
-- Generating done
-- Build files have been written to: /home/edsiper/coding/fluent-bit/build
```
3. Start the compilation process using the `make` command:
```shell
make
```
This command displays results similar to:
```
Scanning dependencies of target msgpack
[ 2%] Building C object lib/msgpack-1.1.0/CMakeFiles/msgpack.dir/src/unpack.c.o
[ 4%] Building C object lib/msgpack-1.1.0/CMakeFiles/msgpack.dir/src/objectc.c.o
[ 7%] Building C object lib/msgpack-1.1.0/CMakeFiles/msgpack.dir/src/version.c.o
...
[ 19%] Building C object lib/monkey/mk_core/CMakeFiles/mk_core.dir/mk_file.c.o
[ 21%] Building C object lib/monkey/mk_core/CMakeFiles/mk_core.dir/mk_rconf.c.o
[ 23%] Building C object lib/monkey/mk_core/CMakeFiles/mk_core.dir/mk_string.c.o
...
Scanning dependencies of target fluent-bit-static
[ 66%] Building C object src/CMakeFiles/fluent-bit-static.dir/flb_pack.c.o
[ 69%] Building C object src/CMakeFiles/fluent-bit-static.dir/flb_input.c.o
[ 71%] Building C object src/CMakeFiles/fluent-bit-static.dir/flb_output.c.o
...
Linking C executable ../bin/fluent-bit
[100%] Built target fluent-bit-bin
```
4. To continue installing the binary on the system, use `make install`:
```shell
make install
```
If the command indicates insufficient permissions, prefix the command with `sudo`.
## Build options
Fluent Bit provides configurable options to CMake that can be enabled or disabled.
### General options
| Option | Description | Default |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------- | ----------------------------- |
| `FLB_ALL` | Enable all features available | `No` |
| `FLB_ARROW` | Build with Apache Arrow support | `No` |
| `FLB_AVRO_ENCODER` | Build with Avro encoding support | `No` |
| `FLB_AWS` | Enable AWS support | `Yes` |
| `FLB_AWS_ERROR_REPORTER` | Build with AWS error reporting support | `No` |
| `FLB_BENCHMARKS` | Enable benchmarks | `No` |
| `FLB_BINARY` | Build executable | `Yes` |
| `FLB_CHUNK_TRACE` | Enable chunk traces | `Yes` |
| `FLB_COVERAGE` | Build with code-coverage | `No` |
| `FLB_CONFIG_YAML` | Enable YAML configuration support | `Yes` |
| `FLB_CORO_STACK_SIZE` | Set coroutine stack size | |
| `FLB_CUSTOM_CALYPTIA` | Enable Calyptia Support | `Yes` |
| `FLB_ENFORCE_ALIGNMENT` | Enable limited platform specific aligned memory access | `No` |
| `FLB_EXAMPLES` | Build examples | `Yes` |
| `FLB_HTTP_SERVER` | Enable HTTP Server | `Yes` |
| `FLB_INOTIFY` | Enable Inotify support | `Yes` |
| `FLB_JEMALLOC` | Build with Jemalloc support | `No` |
| `FLB_KAFKA` | Enable Kafka support | `Yes` |
| `FLB_LUAJIT` | Enable Lua scripting support | `Yes` |
| `FLB_METRICS` | Enable metrics support | `Yes` |
| `FLB_MTRACE` | Enable `mtrace` support | `No` |
| `FLB_PARSER` | Build with Parser support | `Yes` |
| `FLB_POSIX_TLS` | Force POSIX thread storage | `No` |
| `FLB_PROFILES` | Enable profiles support | `Yes` |
| `FLB_PROXY_GO` | Enable Go plugins support | `Yes` |
| `FLB_RECORD_ACCESSOR` | Enable record accessor | `Yes` |
| `FLB_REGEX` | Build with Regex support | `Yes` |
| `FLB_RELEASE` | Build with release mode (`-O2 -g -DNDEBUG`) | `No` |
| `FLB_SHARED_LIB` | Build shared library | `Yes` |
| `FLB_SIGNV4` | Enable AWS Signv4 support | `Yes` |
| `FLB_SIMD` | Enable SIMD support | `No` |
| `FLB_SQLDB` | Enable SQL embedded database support | `Yes` |
| `FLB_STATIC_CONF` | Build binary using static configuration files. The value of this option must be a directory containing configuration files. | |
| `FLB_STREAM_PROCESSOR` | Enable Stream Processor | `Yes` |
| `FLB_TLS` | Build with SSL/TLS support | `Yes` |
| `FLB_UNICODE_ENCODER` | Build with Unicode (`UTF-16LE`, `UTF-16BE`) encoding support | `Yes` (if C++ compiler found) |
| `FLB_UTF8_ENCODER` | Build with UTF8 encoding support | `Yes` |
| `FLB_WASM` | Build with Wasm runtime support | `Yes` |
| `FLB_WASM_STACK_PROTECT` | Build with WASM runtime with strong stack protector flags | `No` |
| `FLB_WAMRC` | Build with Wasm AOT compiler executable | `No` |
| `FLB_WINDOWS_DEFAULTS` | Build with predefined Windows settings | `Yes` |
| `FLB_ZIG` | Enable zig integration | `Yes` |
### Development options
| Option | Description | Default |
| ------------------------- | ---------------------------- | ------- |
| `FLB_BACKTRACE` | Enable stack trace support | `Yes` |
| `FLB_DEBUG` | Build with debug mode (`-g`) | `No` |
| `FLB_SMALL` | Optimize for small size | `No` |
| `FLB_TESTS_INTERNAL` | Enable internal tests | `No` |
| `FLB_TESTS_INTERNAL_FUZZ` | Enable internal fuzz tests | `No` |
| `FLB_TESTS_OSSFUZZ` | Enable OSS-Fuzz build | `No` |
| `FLB_TESTS_RUNTIME` | Enable runtime tests | `No` |
| `FLB_TRACE` | Enable trace mode | `No` |
| `FLB_VALGRIND` | Enable Valgrind support | `No` |
### Optimization options
| Option | Description | Default |
| ----------------------------------------- | -------------------------------------------------------------------------------------------------------- | ------- |
| `FLB_MSGPACK_TO_JSON_INIT_BUFFER_SIZE` | Determine initial buffer size for `msgpack` to `json` conversion in terms of memory used by payload. | `2.0` |
| `FLB_MSGPACK_TO_JSON_REALLOC_BUFFER_SIZE` | Determine percentage of reallocation size when `msgpack` to `json` conversion buffer runs out of memory. | `0.1` |
### Input plugins
Input plugins gather information from a specific source type like network interfaces, some built-in metrics, or through a specific input device.
The following input plugins are available:
| Option | Description | Default |
| -------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- | ------- |
| [`FLB_IN_BLOB`](/manual/data-pipeline/inputs/blob) | Enable Blob input plugin | `On` |
| [`FLB_IN_COLLECTD`](/manual/data-pipeline/inputs/collectd) | Enable Collectd input plugin | `On` |
| [`FLB_IN_CPU`](/manual/data-pipeline/inputs/cpu-metrics) | Enable CPU input plugin | `On` |
| [`FLB_IN_DISK`](/manual/data-pipeline/inputs/disk-io-metrics) | Enable Disk I/O Metrics input plugin | `On` |
| [`FLB_IN_DOCKER`](/manual/data-pipeline/inputs/docker-metrics) | Enable Docker input plugin | `On` |
| [`FLB_IN_DOCKER_EVENTS`](/manual/data-pipeline/inputs/docker-events) | Enable Docker events input plugin | `On` |
| [`FLB_IN_DUMMY`](/manual/data-pipeline/inputs/dummy) | Enable Dummy input plugin | `On` |
| [`FLB_IN_EBPF`](/manual/data-pipeline/inputs/ebpf) | Enable Linux eBPF input plugin | `Off` |
| [`FLB_IN_ELASTICSEARCH`](/manual/data-pipeline/inputs/elasticsearch) | Enable Elasticsearch (Bulk API) input plugin | `On` |
| [`FLB_IN_EXEC`](/manual/data-pipeline/inputs/exec) | Enable Exec input plugin | `On` |
| [`FLB_IN_EXEC_WASI`](/manual/data-pipeline/inputs/exec-wasi) | Enable Exec WASI input plugin | `On` |
| [`FLB_IN_FLUENTBIT_METRICS`](/manual/data-pipeline/inputs/fluentbit-metrics) | Enable Fluent Bit metrics input plugin | `On` |
| [`FLB_IN_FLUENTBIT_LOGS`](/manual/data-pipeline/inputs/fluentbit-logs) | Enable Fluent Bit internal logs input plugin | `On` |
| [`FLB_IN_FORWARD`](/manual/data-pipeline/inputs/forward) | Enable Forward input plugin | `On` |
| [`FLB_IN_GPU_METRICS`](/manual/data-pipeline/inputs/gpu-metrics) | Enable GPU metrics input plugin | `On` |
| [`FLB_IN_HEAD`](/manual/data-pipeline/inputs/head) | Enable Head input plugin | `On` |
| [`FLB_IN_HEALTH`](/manual/data-pipeline/inputs/health) | Enable Health input plugin | `On` |
| [`FLB_IN_HTTP`](/manual/data-pipeline/inputs/http) | Enable HTTP input plugin | `On` |
| [`FLB_IN_KAFKA`](/manual/data-pipeline/inputs/kafka) | Enable Kafka input plugin | `On` |
| [`FLB_IN_KMSG`](/manual/data-pipeline/inputs/kernel-logs) | Enable Kernel log input plugin | `On` |
| [`FLB_IN_KUBERNETES_EVENTS`](/manual/data-pipeline/inputs/kubernetes-events) | Enable Kubernetes Events input plugin | `On` |
| [`FLB_IN_MEM`](/manual/data-pipeline/inputs/memory-metrics) | Enable Memory input plugin | `On` |
| [`FLB_IN_MQTT`](/manual/data-pipeline/inputs/mqtt) | Enable MQTT Broker input plugin | `On` |
| [`FLB_IN_NETIF`](/manual/data-pipeline/inputs/network-io-metrics) | Enable Network I/O metrics input plugin | `On` |
| [`FLB_IN_NGINX_EXPORTER_METRICS`](/manual/data-pipeline/inputs/nginx) | Enable NGINX metrics input plugin | `On` |
| [`FLB_IN_NODE_EXPORTER_METRICS`](/manual/data-pipeline/inputs/node-exporter-metrics) | Enable Node exporter metrics input plugin | `On` |
| [`FLB_IN_OPENTELEMETRY`](/manual/data-pipeline/inputs/opentelemetry) | Enable OpenTelemetry input plugin | `On` |
| [`FLB_IN_PODMAN_METRICS`](/manual/data-pipeline/inputs/podman-metrics) | Enable Podman metrics input plugin | `On` |
| [`FLB_IN_PROC`](/manual/data-pipeline/inputs/process) | Enable Process input plugin | `On` |
| [`FLB_IN_PROCESS_EXPORTER_METRICS`](/manual/data-pipeline/inputs/process-exporter-metrics) | Enable Process exporter metrics input plugin | `On` |
| [`FLB_IN_PROMETHEUS_REMOTE_WRITE`](/manual/data-pipeline/inputs/prometheus-remote-write) | Enable Prometheus remote write input plugin | `On` |
| [`FLB_IN_PROMETHEUS_SCRAPE_METRICS`](/manual/data-pipeline/inputs/prometheus-scrape-metrics) | Enable Prometheus scrape metrics input plugin | `On` |
| [`FLB_IN_PROMETHEUS_TEXTFILE`](/manual/data-pipeline/inputs/prometheus-textfile) | Enable Prometheus textfile input plugin | `On` |
| [`FLB_IN_RANDOM`](/manual/data-pipeline/inputs/random) | Enable Random input plugin | `On` |
| [`FLB_IN_SERIAL`](/manual/data-pipeline/inputs/serial-interface) | Enable Serial input plugin | `On` |
| [`FLB_IN_SPLUNK`](/manual/data-pipeline/inputs/splunk) | Enable Splunk input plugin | `On` |
| [`FLB_IN_STATSD`](/manual/data-pipeline/inputs/statsd) | Enable StatsD input plugin | `On` |
| [`FLB_IN_STDIN`](/manual/data-pipeline/inputs/standard-input) | Enable Standard input plugin | `On` |
| [`FLB_IN_SYSLOG`](/manual/data-pipeline/inputs/syslog) | Enable Syslog input plugin | `On` |
| [`FLB_IN_SYSTEMD`](/manual/data-pipeline/inputs/systemd) | Enable Systemd input plugin | `On` |
| [`FLB_IN_TAIL`](/manual/data-pipeline/inputs/tail) | Enable Tail input plugin | `On` |
| [`FLB_IN_TCP`](/manual/data-pipeline/inputs/tcp) | Enable TCP input plugin | `On` |
| [`FLB_IN_THERMAL`](/manual/data-pipeline/inputs/thermal) | Enable Thermal input plugin | `On` |
| [`FLB_IN_UDP`](/manual/data-pipeline/inputs/udp) | Enable UDP input plugin | `On` |
| [`FLB_IN_WINLOG`](/manual/data-pipeline/inputs/windows-event-log) | Enable Windows Event Log input plugin (Windows Only) | `Off` |
| [`FLB_IN_WINEVTLOG`](/manual/data-pipeline/inputs/windows-event-log-winevtlog) | Enable Windows Event Log input plugin using `winevt.h` API (Windows Only) | `Off` |
| [`FLB_IN_WINDOWS_EXPORTER_METRICS`](/manual/data-pipeline/inputs/windows-exporter-metrics) | Enable Windows exporter metrics input plugin | `On` |
| [`FLB_IN_WINSTAT`](/manual/data-pipeline/inputs/windows-system-statistics) | Enable Windows system statistics input plugin | `Off` |
### Processor plugins
Processor plugins handle the events within the processor pipelines to allow modifying, enriching, or dropping events.
The following table describes the processors available:
| Option | Description | Default |
| ------------------------------------------------------------------------------------------------- | -------------------------------------------------- | ------- |
| [`FLB_PROCESSOR_CONTENT_MODIFIER`](/manual/data-pipeline/processors/content-modifier) | Enable content modifier processor | `On` |
| [`FLB_PROCESSOR_LABELS`](/manual/data-pipeline/processors/labels) | Enable metrics label manipulation processor | `On` |
| [`FLB_PROCESSOR_METRICS_SELECTOR`](/manual/data-pipeline/processors/metrics-selector) | Enable metrics selector processor | `On` |
| [`FLB_PROCESSOR_OPENTELEMETRY_ENVELOPE`](/manual/data-pipeline/processors/opentelemetry-envelope) | Enable OpenTelemetry envelope processor | `On` |
| [`FLB_PROCESSOR_SAMPLING`](/manual/data-pipeline/processors/sampling) | Enable sampling processor | `On` |
| [`FLB_PROCESSOR_SQL`](/manual/data-pipeline/processors/sql) | Enable SQL processor | `On` |
| [`FLB_PROCESSOR_TDA`](/manual/data-pipeline/processors/tda) | Enable Topological Data Analysis (`TDA`) processor | `On` |
### Filter plugins
Filter plugins let you modify, enrich or drop records.
The following table describes the filters available on this version:
| Option | Description | Default |
| ----------------------------------------------------------------------------- | ----------------------------------- | ------- |
| [`FLB_FILTER_AWS`](/manual/data-pipeline/filters/aws-metadata) | Enable AWS metadata filter | `On` |
| [`FLB_FILTER_CHECKLIST`](/manual/data-pipeline/filters/checklist) | Enable Checklist filter | `On` |
| [`FLB_FILTER_ECS`](/manual/data-pipeline/filters/ecs-metadata) | Enable AWS ECS metadata filter | `On` |
| [`FLB_FILTER_EXPECT`](/manual/data-pipeline/filters/expect) | Enable Expect data test filter | `On` |
| [`FLB_FILTER_GIOIP2`](/manual/data-pipeline/filters/geoip2-filter) | Enable Geoip2 filter | `On` |
| [`FLB_FILTER_GREP`](/manual/data-pipeline/filters/grep) | Enable Grep filter | `On` |
| [`FLB_FILTER_KUBERNETES`](/manual/data-pipeline/filters/kubernetes) | Enable Kubernetes metadata filter | `On` |
| [`FLB_FILTER_LOG_TO_METRICS`](/manual/data-pipeline/filters/log_to_metrics) | Enable Log derived metrics filter | `On` |
| [`FLB_FILTER_LUA`](/manual/data-pipeline/filters/lua) | Enable Lua scripting filter | `On` |
| [`FLB_FILTER_MODIFY`](/manual/data-pipeline/filters/modify) | Enable Modify filter | `On` |
| [`FLB_FILTER_MULTILINE`](/manual/data-pipeline/filters/multiline-stacktrace) | Enable Multiline stack trace filter | `On` |
| [`FLB_FILTER_NEST`](/manual/data-pipeline/filters/nest) | Enable Nest filter | `On` |
| [`FLB_FILTER_NIGHTFALL`](/manual/data-pipeline/filters/nightfall) | Enable Nightfall filter | `On` |
| [`FLB_FILTER_PARSER`](/manual/data-pipeline/filters/parser) | Enable Parser filter | `On` |
| [`FLB_FILTER_RECORD_MODIFIER`](/manual/data-pipeline/filters/record-modifier) | Enable Record Modifier filter | `On` |
| [`FLB_FILTER_REWRITE_TAG`](/manual/data-pipeline/filters/rewrite-tag) | Enable Rewrite Tag filter | `On` |
| [`FLB_FILTER_STDOUT`](/manual/data-pipeline/filters/standard-output) | Enable Stdout filter | `On` |
| [`FLB_FILTER_SYSINFO`](/manual/data-pipeline/filters/sysinfo) | Enable Sysinfo filter | `On` |
| [`FLB_FILTER_TENSORFLOW`](/manual/data-pipeline/filters/tensorflow) | Enable Tensorflow filter | `Off` |
| [`FLB_FILTER_THROTTLE`](/manual/data-pipeline/filters/throttle) | Enable Throttle filter | `On` |
| [`FLB_FILTER_TYPE_CONVERTER`](/manual/data-pipeline/filters/type-converter) | Enable Type Converter filter | `On` |
| [`FLB_FILTER_WASM`](/manual/data-pipeline/filters/wasm) | Enable Wasm filter | `On` |
### Output plugins
Output plugins let you flush the information to some external interface, service, or terminal.
The following table describes the output plugins available:
| Option | Description | Default |
| ------------------------------------------------------------------------------------------ | -------------------------------------------------------- | ------- |
| [`FLB_OUT_AZURE`](/manual/data-pipeline/outputs/azure) | Enable Microsoft Azure output plugin | `On` |
| [`FLB_OUT_AZURE_BLOB`](/manual/data-pipeline/outputs/azure_blob) | Enable Microsoft Azure storage blob output plugin | `On` |
| [`FLB_OUT_AZURE_KUSTO`](/manual/data-pipeline/outputs/azure_kusto) | Enable Azure Data Explorer (Kusto) output plugin | `On` |
| [`FLB_OUT_AZURE_LOGS_INGESTION`](/manual/data-pipeline/outputs/azure_logs_ingestion) | Enable Azure Log Ingestion output plugin | `On` |
| [`FLB_OUT_BIGQUERY`](/manual/data-pipeline/outputs/bigquery) | Enable Google BigQuery output plugin | `On` |
| [`FLB_OUT_CHRONICLE`](/manual/data-pipeline/outputs/chronicle) | Enable Google Chronicle output plugin | `On` |
| [`FLB_OUT_CLOUDWATCH_LOGS`](/manual/data-pipeline/outputs/cloudwatch) | Enable Amazon CloudWatch output plugin | `On` |
| [`FLB_OUT_COUNTER`](/manual/data-pipeline/outputs/counter) | Enable Counter output plugin | `On` |
| [`FLB_OUT_DATADOG`](/manual/data-pipeline/outputs/datadog) | Enable Datadog output plugin | `On` |
| [`FLB_OUT_ES`](/manual/data-pipeline/outputs/elasticsearch) | Enable Elastic Search output plugin | `On` |
| [`FLB_OUT_EXIT`](/manual/data-pipeline/outputs/exit) | Enable Exit output plugin | `On` |
| [`FLB_OUT_FILE`](/manual/data-pipeline/outputs/file) | Enable File output plugin | `On` |
| [`FLB_OUT_FLOWCOUNTER`](/manual/data-pipeline/outputs/flowcounter) | Enable Flow counter output plugin | `On` |
| [`FLB_OUT_FORWARD`](/manual/data-pipeline/outputs/forward) | Enable [Fluentd](https://www.fluentd.org) output plugin | `On` |
| [`FLB_OUT_GELF`](/manual/data-pipeline/outputs/gelf) | Enable GELF output plugin | `On` |
| [`FLB_OUT_HTTP`](/manual/data-pipeline/outputs/http) | Enable HTTP output plugin | `On` |
| [`FLB_OUT_INFLUXDB`](/manual/data-pipeline/outputs/influxdb) | Enable InfluxDB output plugin | `On` |
| [`FLB_OUT_KAFKA`](/manual/data-pipeline/outputs/kafka) | Enable Kafka output | `On` |
| [`FLB_OUT_KAFKA_REST`](/manual/data-pipeline/outputs/kafka-rest-proxy) | Enable Kafka REST Proxy output plugin | `On` |
| [`FLB_OUT_KINESIS_FIREHOSE`](/manual/data-pipeline/outputs/firehose) | Enable Amazon Kinesis Data Firehose output plugin | `On` |
| [`FLB_OUT_KINESIS_STREAMS`](/manual/data-pipeline/outputs/kinesis) | Enable Amazon Kinesis Data Streams output plugin | `On` |
| `FLB_OUT_LIB` | Enable Library output plugin | `On` |
| [`FLB_OUT_LOGDNA`](/manual/data-pipeline/outputs/logdna) | Enable LogDNA output plugin | `On` |
| [`FLB_OUT_LOKI`](/manual/data-pipeline/outputs/loki) | Enable Loki output plugin | `On` |
| [`FLB_OUT_NATS`](/manual/data-pipeline/outputs/nats) | Enable NATS output plugin | `On` |
| [`FLB_OUT_NRLOGS`](/manual/data-pipeline/outputs/new-relic) | Enable New Relic output plugin | `On` |
| [`FLB_OUT_NULL`](/manual/data-pipeline/outputs/null) | Enable `NULL` output plugin | `On` |
| [`FLB_OUT_OPENSEARCH`](/manual/data-pipeline/outputs/opensearch) | Enable OpenSearch output plugin | `On` |
| [`FLB_OUT_OPENTELEMETRY`](/manual/data-pipeline/outputs/opentelemetry) | Enable OpenTelemetry output plugin | `On` |
| [`FLB_OUT_ORACLE_LOG_ANALYTICS`](/manual/data-pipeline/outputs/oci-logging-analytics) | Enable Oracle Cloud Infrastructure Logging output plugin | `On` |
| [`FLB_OUT_PGSQL`](/manual/data-pipeline/outputs/postgresql) | Enable PostgreSQL output plugin | `Off` |
| [`FLB_OUT_PLOT`](/manual/data-pipeline/outputs/plot) | Enable Plot output plugin | `On` |
| [`FLB_OUT_PROMETHEUS_EXPORTER`](/manual/data-pipeline/outputs/prometheus-exporter) | Enable Prometheus exporter output plugin | `On` |
| [`FLB_OUT_PROMETHEUS_REMOTE_WRITE`](/manual/data-pipeline/outputs/prometheus-remote-write) | Enable Prometheus remote write output plugin | `On` |
| [`FLB_OUT_S3`](/manual/data-pipeline/outputs/s3) | Enable Amazon S3 output plugin | `On` |
| [`FLB_OUT_SKYWALKING`](/manual/data-pipeline/outputs/skywalking) | Enable Apache Skywalking output plugin | `On` |
| [`FLB_OUT_SLACK`](/manual/data-pipeline/outputs/slack) | Enable Slack output plugin | `On` |
| [`FLB_OUT_SPLUNK`](/manual/data-pipeline/outputs/splunk) | Enable Splunk output plugin | `On` |
| [`FLB_OUT_STACKDRIVER`](/manual/data-pipeline/outputs/stackdriver) | Enable Stackdriver output plugin | `On` |
| [`FLB_OUT_STDOUT`](/manual/data-pipeline/outputs/standard-output) | Enable STDOUT output plugin | `On` |
| [`FLB_OUT_SYSLOG`](/manual/data-pipeline/outputs/syslog) | Enable Syslog output plugin | `On` |
| [`FLB_OUT_TD`](/manual/data-pipeline/outputs/treasure-data) | Enable Treasure Data output plugin | `On` |
| [`FLB_OUT_TCP`](/manual/data-pipeline/outputs/tcp-and-tls) | Enable TCP/TLS output plugin | `On` |
| [`FLB_OUT_UDP`](/manual/data-pipeline/outputs/udp) | Enable UDP output plugin | `On` |
| [`FLB_OUT_VIVO_EXPORTER`](/manual/data-pipeline/outputs/vivo-exporter) | Enable Vivo exporter output plugin | `On` |
| [`FLB_OUT_WEBSOCKET`](/manual/data-pipeline/outputs/websocket) | Enable WebSocket output plugin | `On` |
# Build with static configuration
[Fluent Bit](https://fluentbit.io) in normal operation mode is configurable through text files or using specific arguments in the command line. Although this is the ideal deployment case, there are scenarios where a more restricted configuration is required. Static configuration mode restricts configuration ability.
Static configuration mode includes a built-in configuration in the final binary of Fluent Bit, disabling the usage of external files or flags at runtime.
## Get started
### Requirements
The following steps assume you are familiar with configuring Fluent Bit using text files and you have experience building it from scratch as described in [Build and Install](/manual/installation/downloads/source/build-and-install).
#### Configuration directory
In your file system, prepare a specific directory that will be used as an entry point for the build system to lookup and parse the configuration files. This directory must contain a minimum of one configuration file, called `fluent-bit.conf`, that contains the required [`SERVICE`](/manual/administration/configuring-fluent-bit/yaml/service-section), [`INPUT`](/manual/data-pipeline/inputs), and [`OUTPUT`](/manual/data-pipeline/outputs) sections.
As an example, create a new `fluent-bit.yaml` file or `fluent-bit.conf` file:
{% tabs %}
{% tab title="fluent-bit.yaml" %}
```yaml
service:
flush: 1
daemon: off
log_level: info
pipeline:
inputs:
- name: cpu
outputs:
- name: stdout
match: '*'
```
{% endtab %}
{% tab title="fluent-bit.conf" %}
```
[SERVICE]
Flush 1
Daemon off
Log_Level info
[INPUT]
Name cpu
[OUTPUT]
Name stdout
Match *
```
{% endtab %}
{% endtabs %}
This configuration calculates CPU metrics from the running system and prints them to the standard output interface.
#### Build with custom configuration
1. Go to the Fluent Bit source code build directory:
```shell
cd fluent-bit/build/
```
2. Run CMake, appending the `FLB_STATIC_CONF` option pointing to the configuration directory recently created:
```shell
cmake -DFLB_STATIC_CONF=/path/to/my/confdir/
```
3. Build Fluent Bit:
```shell
make
```
The generated `fluent-bit` binary is ready to run without additional configuration:
```shell
$ bin/fluent-bit
...
[0] cpu.local: [1539984752.000347547, {"cpu_p"=>0.750000, "user_p"=>0.500000, "system_p"=>0.250000, "cpu0.p_cpu"=>1.000000, "cpu0.p_user"=>1.000000, "cpu0.p_system"=>0.000000, "cpu1.p_cpu"=>0.000000, "cpu1.p_user"=>0.000000, "cpu1.p_system"=>0.000000, "cpu2.p_cpu"=>0.000000, "cpu2.p_user"=>0.000000, "cpu2.p_system"=>0.000000, "cpu3.p_cpu"=>1.000000, "cpu3.p_user"=>1.000000, "cpu3.p_system"=>0.000000}]
```
# Linux packages
Fluent Bit is available for a variety of Linux distributions and embedded Linux systems.
The most secure option is to create the repositories according to the instructions for your specific OS.
## Single line install
An installation script is provided for use with most Linux targets. This will by default install the most recent version released.
```bash
curl https://raw.githubusercontent.com/fluent/fluent-bit/master/install.sh | sh
```
This is a helper and should always be validated prior to use.
## GPG key updates
For the 1.9.0 and 1.8.15 releases and later, the GPG key [has been updated](https://packages.fluentbit.io/fluentbit.key). Ensure the new key is added.
The GPG Key fingerprint of the new key is:
```
C3C0 A285 34B9 293E AF51 FABD 9F9D DC08 3888 C1CD
Fluentbit releases (Releases signing key)
```
The previous key is [still available](https://packages.fluentbit.io/fluentbit-legacy.key) and might be required to install previous versions.
The GPG Key fingerprint of the old key is:
```
F209 D876 2A60 CD49 E680 633B 4FF8 368B 6EA0 722A
```
Refer to the [supported platform documentation](/manual/installation/downloads#supported-platforms-and-packages) to see which platforms are supported in each release.
## Migration to Fluent Bit
For version 1.9 and later, `td-agent-bit` is a deprecated package and is removed after 1.9.9. The correct package name to use now is `fluent-bit`.
# Amazon Linux
Fluent Bit is distributed as the `fluent-bit` package and is available for Amazon Linux 2 and Amazon Linux 2023. The following architectures are supported:
* x86\_64
* aarch64 / arm64v8
## Install on Amazon EC2
To install Fluent Bit and related AWS output plugins on Amazon Linux 2 on EC2 using AWS Systems Manager, follow [this AWS guide](https://github.com/aws/aws-for-fluent-bit/tree/mainline/examples/fluent-bit/systems-manager-ec2).
## General installation
To install Fluent Bit on any Amazon Linux instance, follow these steps.
1. Fluent Bit is provided through a Yum repository. To add the repository reference to your system, add a new file called `fluent-bit.repo` in `/etc/yum.repos.d/` with the following content:
{% tabs %}
{% tab title="Amazon Linux 2" %}
```
[fluent-bit]
name = Fluent Bit
baseurl = https://packages.fluentbit.io/amazonlinux/2/
gpgcheck=1
gpgkey=https://packages.fluentbit.io/fluentbit.key
enabled=1
```
{% endtab %}
{% tab title="Amazon Linux 2023" %}
```
[fluent-bit]
name = Fluent Bit
baseurl = https://packages.fluentbit.io/amazonlinux/2023/
gpgcheck=1
gpgkey=https://packages.fluentbit.io/fluentbit.key
enabled=1
```
{% endtab %}
{% endtabs %}
{% hint style="info" %}
You should always enable `gpgcheck` for security reasons. All Fluent Bit packages are signed.
{% endhint %}
2. Ensure your [GPG key](/manual/installation/downloads/linux#gpg-key-updates) is up to date.
3. After your repository is configured, run the following command to install it:
```shell
sudo yum install fluent-bit
```
4. Instruct `systemd` to enable the service:
```shell
sudo systemctl start fluent-bit
```
If you do a status check, you should see a similar output like this:
```shell
$ systemctl status fluent-bit
● fluent-bit.service - Fluent Bit
Loaded: loaded (/usr/lib/systemd/system/fluent-bit.service; disabled; vendor preset: disabled)
Active: active (running) since Thu 2016-07-07 02:08:01 BST; 9s ago
Main PID: 3820 (fluent-bit)
CGroup: /system.slice/fluent-bit.service
└─3820 /opt/fluent-bit/bin/fluent-bit -c /etc/fluent-bit/fluent-bit.conf
...
```
The default Fluent Bit configuration collect metrics of CPU usage and sends the records to the standard output. You can see the outgoing data in your `/var/log/messages` file.
# Buildroot embedded Linux
Install Fluent Bit in your embedded Linux system.
## Install
To install, select Fluent Bit in your `defconfig`. See the `Config.in` file for all configuration options.
```
BR2_PACKAGE_FLUENT_BIT=y
```
## Run
The default configuration file is written to:
```
/etc/fluent-bit/fluent-bit.conf
```
Fluent Bit is started by the `S99fluent-bit` script.
## Support
All configurations with a tool chain that supports threads and dynamic library linking are supported.
# Debian
Fluent Bit is distributed as the `fluent-bit` package and is available for the latest stable Debian system.
The following architectures are supported
* `x86_64`
* `aarch64`
* `arm64v8`
The recommended secure deployment approach is to use the following instructions:
## Server GPG key
The first step is to add the Fluent Bit server GPG key to your keyring to ensure you can get the correct signed packages.
Follow the official [Debian wiki guidance](https://wiki.debian.org/DebianRepository/UseThirdParty#OpenPGP_Key_distribution).
```shell
sudo sh -c 'curl https://packages.fluentbit.io/fluentbit.key | gpg --dearmor > /usr/share/keyrings/fluentbit-keyring.gpg'
```
## Update your sources lists
For Debian, you must add the Fluent Bit APT server entry to your sources lists. Ensure codename is set to your specific [Debian release name](https://wiki.debian.org/DebianReleases#Production_Releases). (for example: `bookworm` for Debian 12).
```shell
codename=$(grep -oP '(?<=VERSION_CODENAME=).*' /etc/os-release 2>/dev/null || lsb_release -cs 2>/dev/null)
```
Update your source's lists:
```bash
echo "deb [signed-by=/usr/share/keyrings/fluentbit-keyring.gpg] https://packages.fluentbit.io/debian/$codename $codename main" | sudo tee /etc/apt/sources.list.d/fluent-bit.list
```
## Update your repositories database
Update your system's `apt` database:
```shell
sudo apt-get update
```
{% hint style="info" %}
Fluent Bit recommends upgrading your system (`sudo apt-get upgrade`). This could avoid potential issues with expired certificates.
{% endhint %}
## Install Fluent Bit
1. Ensure your [GPG key](/manual/installation/downloads/linux#gpg-key-updates) is up to date.
2. Use the following `apt-get` command to install the latest Fluent Bit:
```shell
sudo apt-get install fluent-bit
```
3. Instruct `systemd` to enable the service:
```shell
sudo systemctl start fluent-bit
```
If you do a status check, you should see a similar output similar to:
```shell
$ sudo service fluent-bit status
● fluent-bit.service - Fluent Bit
Loaded: loaded (/lib/systemd/system/fluent-bit.service; disabled; vendor preset: enabled)
Active: active (running) since mié 2016-07-06 16:58:25 CST; 2h 45min ago
Main PID: 6739 (fluent-bit)
Tasks: 1
Memory: 656.0K
CPU: 1.393s
CGroup: /system.slice/fluent-bit.service
└─6739 /opt/fluent-bit/bin/fluent-bit -c /etc/fluent-bit/fluent-bit.conf
...
```
The default Fluent Bit configuration collect metrics of CPU usage and sends the records to the standard output. You can see the outgoing data in your `/var/log/messages` file.
# Raspbian and Raspberry Pi
Fluent Bit is distributed as the `fluent-bit` package and is available for [Raspberry Pi](https://www.raspberrypi.com/software/operating-systems/). The following versions are supported:
* Raspbian Bookworm (12)
* Raspbian Bullseye (11)
* Raspbian Buster (10)
## Server GPG key
The first step is to add the Fluent Bit server GPG key to your keyring so you can get Fluent Bit signed packages:
```shell
sudo sh -c 'curl https://packages.fluentbit.io/fluentbit.key | sudo apt-key add - '
```
## Update your sources lists
On Debian and derivative systems such as Raspbian, you need to add the Fluent Bit APT server entry to your sources lists.
Add the following content at bottom of your `/etc/apt/sources.list` file.
### Raspbian 12 (Bookworm)
```shell
echo "deb https://packages.fluentbit.io/raspbian/bookworm bookworm main" | sudo tee /etc/apt/sources.list.d/fluent-bit.list
```
### Raspbian 11 (Bullseye)
```shell
echo "deb https://packages.fluentbit.io/raspbian/bullseye bullseye main" | sudo tee /etc/apt/sources.list.d/fluent-bit.list
```
### Raspbian 10 (Buster)
```shell
echo "deb https://packages.fluentbit.io/raspbian/buster buster main" | sudo tee /etc/apt/sources.list.d/fluent-bit.list
```
### Update your repositories database
Now let your system update the `apt` database:
```shell
sudo apt-get update
```
{% hint style="info" %}
Fluent Bit recommends upgrading your system (`sudo apt-get upgrade`) to avoid potential issues with expired certificates.
{% endhint %}
## Install Fluent Bit
1. Ensure your [GPG key](/manual/installation/downloads/linux#gpg-key-updates) is up to date.
2. Use the following `apt-get` command to install the latest Fluent Bit:
```shell
sudo apt-get install fluent-bit
```
3. Instruct `systemd` to enable the service:
```shell
sudo service fluent-bit start
```
If you do a status check, you should see a similar output like this:
```shell
$ sudo service fluent-bit status
● fluent-bit.service - Fluent Bit
Loaded: loaded (/lib/systemd/system/fluent-bit.service; disabled; vendor preset: enabled)
Active: active (running) since mié 2016-07-06 16:58:25 CST; 2h 45min ago
Main PID: 6739 (fluent-bit)
Tasks: 1
Memory: 656.0K
CPU: 1.393s
CGroup: /system.slice/fluent-bit.service
└─6739 /opt/fluent-bit/bin/fluent-bit -c /etc/fluent-bit/fluent-bit.conf
...
```
The default configuration of Fluent Bit collects metrics for CPU usage and sends the records to the standard output. You can see the outgoing data in your `/var/log/syslog` file.
# Red Hat and CentOS
Fluent Bit is distributed as the `fluent-bit` package and is available for the latest stable CentOS system.
Fluent Bit supports the following architectures:
* `x86_64`
* `aarch64`
* `arm64v8`
For CentOS 9 and later, Fluent Bit uses [CentOS Stream](https://www.centos.org/centos-stream/) as the canonical base system.
The recommended secure deployment approach is to use the following instructions:
## CentOS 8
CentOS 8 is now end-of-life, so the default Yum repositories are unavailable.
Ensure you've configured an appropriate mirror. For example:
```shell
$ sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-* && \
$ sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.centos.org|g' /etc/yum.repos.d/CentOS-*
```
An alternative is to use Rocky or Alma Linux, which should be equivalent.
## RHEL, AlmaLinux, RockyLinux, and CentOS 9 Stream
From CentOS 9 Stream and later, the CentOS dependencies will update more often than downstream usage. This might mean that incompatible (more recent) versions are provided of certain dependencies (for example, OpenSSL). For OSS, Fluent Bit also provide RockyLinux and AlmaLinux repositories.
Replace the `centos` string in Yum configuration with `almalinux` or `rockylinux` to use those repositories instead. This might be required for RHEL 9 as well which will no longer track equivalent CentOS 9 stream dependencies. No RHEL 9 build is provided, as it's expected you're using one of the OSS variants listed.
## Configure YUM
The`fluent-bit` package is provided through a Yum repository. To add the repository reference to your system:
1. In `/etc/yum.repos.d/`, add a new file called `fluent-bit.repo`.
2. Add the following content to the file:
```
[fluent-bit]
name=Fluent Bit
baseurl=https://packages.fluentbit.io/centos/$releasever/$basearch/
gpgcheck=1
gpgkey=https://packages.fluentbit.io/fluentbit.key
repo_gpgcheck=1
enabled=1
```
3. As a best practice, enable `gpgcheck` and `repo_gpgcheck` for security reasons. Fluent Bit signs its repository metadata and all Fluent Bit packages.
### Install
1. Ensure your [GPG key](/manual/installation/downloads/linux#gpg-key-updates) is up to date.
2. After your repository is configured, run the following command to install it:
```shell
sudo yum install fluent-bit
```
3. Instruct `Systemd` to enable the service:
```shell
sudo systemctl start fluent-bit
```
If you do a status check, you should see a similar output like this:
```shell
$ systemctl status fluent-bit
● fluent-bit.service - Fluent Bit
Loaded: loaded (/usr/lib/systemd/system/fluent-bit.service; disabled; vendor preset: disabled)
Active: active (running) since Thu 2016-07-07 02:08:01 BST; 9s ago
Main PID: 3820 (fluent-bit)
CGroup: /system.slice/fluent-bit.service
└─3820 /opt/fluent-bit/bin/fluent-bit -c etc/fluent-bit/fluent-bit.conf
...
```
The default Fluent Bit configuration collect metrics of CPU usage and sends the records to the standard output. You can see the outgoing data in your `/var/log/messages` file.
## FAQ
### Yum install fails with a `404 - Page not found` error for the package mirror
The `fluent-bit.repo` file for the latest installations of Fluent Bit uses a `$releasever` variable to determine the correct version of the package to install to your system:
```
[fluent-bit]
name=Fluent Bit
baseurl=https://packages.fluentbit.io/centos/$releasever/$basearch/
```
Depending on your Red Hat distribution version, this variable can return a value other than the OS major release version (for example, RHEL7 Server distributions return `7Server` instead of `7`). The Fluent Bit package URL uses the major OS release version, so any other value here will cause a 404.
To resolve this issue, replace the `$releasever` variable with your system's OS major release version. For example:
```
[fluent-bit]
name=Fluent Bit
baseurl=https://packages.fluentbit.io/centos/7/$basearch/
gpgcheck=1
gpgkey=https://packages.fluentbit.io/fluentbit.key
repo_gpgcheck=1
enabled=1
```
### Yum install fails with incompatible dependencies using CentOS 9+
CentOS 9 and later will no longer be compatible with RHEL 9 as it might track more recent dependencies. Alternative AlmaLinux and RockyLinux repositories are available.
See the previous guidance.
# Rocky Linux and Alma Linux
Fluent Bit is distributed as the `fluent-bit` package and is available for the latest versions of Rocky or Alma Linux now that CentOS Stream is tracking more recent dependencies.
Fluent Bit supports the following architectures:
* `x86_64`
* `aarch64`
* `arm64v8`
## RHEL 9
From CentOS 9 Stream and later, the CentOS dependencies will update more often than downstream usage. This might mean that incompatible (more recent) versions are provided of certain dependencies (for example, OpenSSL). For OSS, there are RockyLinux and AlmaLinux repositories. This might be required for RHEL 9 as well which will no longer track equivalent CentOS 9 stream dependencies. No RHEL 9 build is provided, it is expected to use one of the OSS variants listed.
## Configure YUM
The `fluent-bit` package is provided through a Yum repository. To add the repository reference to your system:
1. In `/etc/yum.repos.d/`, add a new file called `fluent-bit.repo`.
2. Add the following content to the file - replace `almalinux` with `rockylinux` if required:
```
[fluent-bit]
name = Fluent Bit
baseurl = https://packages.fluentbit.io/almalinux/$releasever/
gpgcheck=1
gpgkey=https://packages.fluentbit.io/fluentbit.key
repo_gpgcheck=1
enabled=1
```
3. As a best practice, enable `gpgcheck` and `repo_gpgcheck` for security reasons. Fluent Bit signs its repository metadata and all Fluent Bit packages.
## Install
1. After your repository is configured, run the following command to install it:
```shell
sudo yum install fluent-bit
```
2. Instruct `Systemd` to enable the service:
```shell
sudo systemctl start fluent-bit
```
If you do a status check, you should see a similar output like this:
```shell
$ systemctl status fluent-bit
● fluent-bit.service - Fluent Bit
Loaded: loaded (/usr/lib/systemd/system/fluent-bit.service; disabled; vendor preset: disabled)
Active: active (running) since Thu 2016-07-07 02:08:01 BST; 9s ago
Main PID: 3820 (fluent-bit)
CGroup: /system.slice/fluent-bit.service
└─3820 /opt/fluent-bit/bin/fluent-bit -c etc/fluent-bit/fluent-bit.conf
...
```
The default Fluent Bit configuration collect metrics of CPU usage and sends the records to the standard output. You can see the outgoing data in your `/var/log/messages` file.
# Ubuntu
Fluent Bit is distributed as the `fluent-bit` package and is available for long-term support releases of Ubuntu. The latest officially supported version is Ubuntu 26.04. Long-term support releases are also available for Noble Numbat (24.04) and Jammy Jellyfish (22.04).
The recommended secure deployment approach is to use the following instructions.
## Server GPG key
Add the Fluent Bit server GPG key to your keyring to ensure you can get the correct signed packages.
Follow the official [Debian wiki guidance](https://wiki.debian.org/DebianRepository/UseThirdParty#OpenPGP_Key_distribution).
```shell
sudo sh -c 'curl https://packages.fluentbit.io/fluentbit.key | gpg --dearmor > /usr/share/keyrings/fluentbit-keyring.gpg'
```
## Update your sources lists
On Ubuntu, you need to add the Fluent Bit APT server entry to your sources lists. Ensure `codename` is set to your specific [Ubuntu release name](https://wiki.ubuntu.com/Releases). For example, `focal` for Ubuntu 20.04.
```shell
codename=$(grep -oP '(?<=VERSION_CODENAME=).*' /etc/os-release 2>/dev/null || lsb_release -cs 2>/dev/null)
```
Update your source's list:
```shell
echo "deb [signed-by=/usr/share/keyrings/fluentbit-keyring.gpg] https://packages.fluentbit.io/ubuntu/$codename $codename main" | sudo tee /etc/apt/sources.list.d/fluent-bit.list
```
### Update your repositories database
Update the `apt` database on your system:
```shell
sudo apt-get update
```
{% hint style="info" %}
Fluent Bit recommends upgrading your system to avoid potential issues with expired certificates:
`sudo apt-get upgrade`
If you receive the error `Certificate verification failed`, check if the package `ca-certificates` is properly installed:
`sudo apt-get install ca-certificates`
{% endhint %}
## Install Fluent Bit
1. Ensure your [GPG key](/manual/installation/downloads/linux#gpg-key-updates) is up to date.
2. Use the following `apt-get` command to install the latest Fluent Bit:
```shell
sudo apt-get install fluent-bit
```
3. Instruct `systemd` to enable the service:
```shell
sudo systemctl start fluent-bit
```
If you do a status check, you should see a similar output like this:
```shell
$ systemctl status fluent-bit
● fluent-bit.service - Fluent Bit
Loaded: loaded (/lib/systemd/system/fluent-bit.service; disabled; vendor preset: enabled)
Active: active (running) since mié 2016-07-06 16:58:25 CST; 2h 45min ago
Main PID: 6739 (fluent-bit)
Tasks: 1
Memory: 656.0K
CPU: 1.393s
CGroup: /system.slice/fluent-bit.service
└─6739 /opt/fluent-bit/bin/fluent-bit -c /etc/fluent-bit/fluent-bit.conf
...
```
The default configuration of `fluent-bit` is collecting metrics of CPU usage and sending the records to the standard output. You can see the outgoing data in your `/var/log/syslog` file.
# Yocto embedded Linux
[Fluent Bit](https://fluentbit.io) source code provides BitBake recipes to configure, build, and package the software for a Yocto-based image. Specific steps in the usage of these recipes in your Yocto environment (Poky) is out of the scope of this documentation.
Fluent Bit distributes two main recipes, one for testing/dev purposes and one with the latest stable release.
| Version | Recipe | Description |
| --------- | ----------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
| `devel` | [fluent-bit\_git.bb](https://github.com/fluent/fluent-bit/blob/master/fluent-bit_git.bb) | Build Fluent Bit from Git master. Use for development and testing purposes only. |
| `v1.8.11` | [fluent-bit\_1.8.11.bb](https://github.com/fluent/fluent-bit/blob/v1.8.11/fluent-bit_1.8.11.bb) | Build latest stable version of Fluent Bit. |
It's strongly recommended to always use the stable release of the Fluent Bit recipe and not the one from Git master for production deployments.
## Fluent Bit and other architectures
Fluent Bit 1.1.x and later fully supports `x86_64`, `x86`, `arm32v7`, and `arm64v8`.
# Containers on AWS
AWS maintains a distribution of Fluent Bit that combines the latest official release with a set of Go Plugins for sending logs to AWS services. AWS and Fluent Bit are working together to rewrite their plugins for inclusion in the official Fluent Bit distribution.
## Plugins
The [AWS for Fluent Bit](https://github.com/aws/aws-for-fluent-bit) image contains Go Plugins for:
* Amazon CloudWatch as `cloudwatch_logs`. See the [Fluent Bit docs](/manual/data-pipeline/outputs/cloudwatch) or the [Plugin repository](https://github.com/aws/amazon-cloudwatch-logs-for-fluent-bit).
* Amazon Kinesis Data Firehose as `kinesis_firehose`. See the [Fluent Bit docs](/manual/data-pipeline/outputs/firehose) or the [Plugin repository](https://github.com/aws/amazon-kinesis-firehose-for-fluent-bit).
* Amazon Kinesis Data Streams as `kinesis_streams`. See the [Fluent Bit docs](/manual/data-pipeline/outputs/kinesis) or the [Plugin repository](https://github.com/aws/amazon-kinesis-streams-for-fluent-bit).
These plugins are higher performance than Go plugins.
Also, Fluent Bit includes an S3 output plugin named `s3`.
* [Amazon S3](/manual/data-pipeline/outputs/s3)
## Versions and regional repositories
AWS vends their container image using [Docker Hub](https://hub.docker.com/r/amazon/aws-for-fluent-bit), and a set of highly available regional Amazon ECR repositories. For more information, see the [AWS for Fluent Bit GitHub repository](https://github.com/aws/aws-for-fluent-bit#public-images).
The AWS for Fluent Bit image uses a custom versioning scheme because it contains multiple projects. To see what each release contains, see the [release notes on GitHub](https://github.com/aws/aws-for-fluent-bit/releases).
## SSM public parameters
AWS vends SSM public parameters with the regional repository link for each image. These parameters can be queried by any AWS account.
To see a list of available version tags in a given region, run the following command:
```shell
aws ssm get-parameters-by-path --region eu-central-1 --path /aws/service/aws-for-fluent-bit/ --query 'Parameters[*].Name'
```
To see the ECR repository URI for a given image tag in a given region, run the following:
```shell
aws ssm get-parameter --region ap-northeast-1 --name /aws/service/aws-for-fluent-bit/2.0.0
```
You can use these SSM public parameters as parameters in your CloudFormation templates:
```
Parameters:
FireLensImage:
Description: Fluent Bit image for the FireLens Container
Type: AWS::SSM::Parameter::Value
Default: /aws/service/aws-for-fluent-bit/latest
```
# Docker
Fluent Bit container images are available on Docker Hub ready for production usage. Current available images can be deployed in multiple architectures.
## Start Docker
Use the following command to start Docker with Fluent Bit:
```shell
docker run -ti cr.fluentbit.io/fluent/fluent-bit
```
### Use a configuration file
Use the following command to start Fluent Bit while using a configuration file:
{% tabs %}
{% tab title="fluent-bit.conf" %}
```shell
docker run -ti -v ./fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf \
cr.fluentbit.io/fluent/fluent-bit
```
{% endtab %}
{% tab title="fluent-bit.yaml" %}
```shell
docker run -ti -v ./fluent-bit.yaml:/fluent-bit/etc/fluent-bit.yaml \
cr.fluentbit.io/fluent/fluent-bit \
-c /fluent-bit/etc/fluent-bit.yaml
```
{% endtab %}
{% endtabs %}
## Tags and versions
The following table describes the Linux container tags that are available on Docker Hub [`fluent/fluent-bit`](https://hub.docker.com/r/fluent/fluent-bit/) repository:
| Tags | Manifest Architectures | Description |
| ------------ | --------------------------- | -------------------------------------------------------------- |
| 5.0.8-debug | amd64, arm64, arm/v7 | Debug images |
| 5.0.8 | amd64, arm64, arm/v7 | Release [v5.0.8](https://fluentbit.io/announcements/v5.0.8/) |
| 5.0.7-debug | amd64, arm64, arm/v7 | Debug images |
| 5.0.7 | amd64, arm64, arm/v7 | Release [v5.0.7](https://fluentbit.io/announcements/v5.0.7/) |
| 5.0.6-debug | amd64, arm64, arm/v7 | Debug images |
| 5.0.6 | amd64, arm64, arm/v7 | Release [v5.0.6](https://fluentbit.io/announcements/v5.0.6/) |
| 5.0.3-debug | amd64, arm64, arm/v7 | Debug images |
| 5.0.3 | amd64, arm64, arm/v7 | Release [v5.0.3](https://fluentbit.io/announcements/v5.0.3/) |
| 5.0.2-debug | amd64, arm64, arm/v7 | Debug images |
| 5.0.2 | amd64, arm64, arm/v7 | Release [v5.0.2](https://fluentbit.io/announcements/v5.0.2/) |
| 4.2.3-debug | amd64, arm64, arm/v7 | Debug images |
| 4.2.3 | amd64, arm64, arm/v7 | Release [v4.2.3](https://fluentbit.io/announcements/v4.2.3/) |
| 4.2.2-debug | amd64, arm64, arm/v7 | Debug images |
| 4.2.2 | amd64, arm64, arm/v7 | Release [v4.2.2](https://fluentbit.io/announcements/v4.2.2/) |
| 4.2.1-debug | amd64, arm64, arm/v7 | Debug images |
| 4.2.1 | amd64, arm64, arm/v7 | Release [v4.2.1](https://fluentbit.io/announcements/v4.2.1/) |
| 4.2.0-debug | amd64, arm64, arm/v7 | Debug images |
| 4.2.0 | amd64, arm64, arm/v7 | Release [v4.2.0](https://fluentbit.io/announcements/v4.2.0/) |
| 4.1.2-debug | amd64, arm64, arm/v7 | Debug images |
| 4.1.2 | amd64, arm64, arm/v7 | Release v4.1.2 |
| 4.1.1-debug | amd64, arm64, arm/v7 | Debug images |
| 4.1.1 | amd64, arm64, arm/v7 | Release [v4.1.1](https://fluentbit.io/announcements/v4.1.1/) |
| 4.1.0-debug | amd64, arm64, arm/v7 | Debug images |
| 4.1.0 | amd64, arm64, arm/v7 | Release [v4.1.0](https://fluentbit.io/announcements/v4.1.0/) |
| 4.0.12-debug | amd64, arm64, arm/v7 | Debug images |
| 4.0.12 | amd64, arm64, arm/v7 | Release [v4.0.12](https://fluentbit.io/announcements/v4.0.12/) |
| 4.0.11-debug | amd64, arm64, arm/v7 | Debug images |
| 4.0.11 | amd64, arm64, arm/v7 | Release [v4.0.11](https://fluentbit.io/announcements/v4.0.11/) |
| 4.0.10-debug | amd64, arm64, arm/v7 | Debug images |
| 4.0.10 | amd64, arm64, arm/v7 | Release [v4.0.10](https://fluentbit.io/announcements/v4.0.10/) |
| 4.0.9-debug | amd64, arm64, arm/v7 | Debug images |
| 4.0.9 | amd64, arm64, arm/v7 | Release [v4.0.9](https://fluentbit.io/announcements/v4.0.9/) |
| 4.0.8-debug | amd64, arm64, arm/v7 | Debug images |
| 4.0.8 | amd64, arm64, arm/v7 | Release [v4.0.8](https://fluentbit.io/announcements/v4.0.8/) |
| 4.0.7-debug | amd64, arm64, arm/v7 | Debug images |
| 4.0.7 | amd64, arm64, arm/v7 | Release [v4.0.7](https://fluentbit.io/announcements/v4.0.7/) |
| 4.0.6-debug | amd64, arm64, arm/v7 | Debug images |
| 4.0.6 | amd64, arm64, arm/v7 | Release [v4.0.6](https://fluentbit.io/announcements/v4.0.6/) |
| 4.0.5-debug | amd64, arm64, arm/v7 | Debug images |
| 4.0.5 | amd64, arm64, arm/v7 | Release [v4.0.5](https://fluentbit.io/announcements/v4.0.5/) |
| 4.0.4-debug | amd64, arm64, arm/v7 | Debug images |
| 4.0.4 | amd64, arm64, arm/v7 | Release [v4.0.4](https://fluentbit.io/announcements/v4.0.4/) |
| 4.0.3-debug | amd64, arm64, arm/v7 | Debug images |
| 4.0.3 | amd64, arm64, arm/v7 | Release [v4.0.3](https://fluentbit.io/announcements/v4.0.3/) |
| 4.0.1-debug | amd64, arm64, arm/v7 | Debug images |
| 4.0.1 | amd64, arm64, arm/v7 | Release [v4.0.1](https://fluentbit.io/announcements/v4.0.1/) |
| 4.0.0-debug | amd64, arm64, arm/v7 | Debug images |
| 4.0.0 | amd64, arm64, arm/v7 | Release [v4.0.0](https://fluentbit.io/announcements/v4.0.0/) |
| 3.2.10-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.2.10 | amd64, arm64, arm/v7, s390x | Release [v3.2.10](https://fluentbit.io/announcements/v3.2.10/) |
| 3.2.9-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.2.9 | amd64, arm64, arm/v7, s390x | Release [v3.2.9](https://fluentbit.io/announcements/v3.2.9/) |
| 3.2.8-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.2.8 | amd64, arm64, arm/v7, s390x | Release [v3.2.8](https://fluentbit.io/announcements/v3.2.8/) |
| 3.2.7-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.2.7 | amd64, arm64, arm/v7, s390x | Release [v3.2.7](https://fluentbit.io/announcements/v3.2.7/) |
| 3.2.6-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.2.6 | amd64, arm64, arm/v7, s390x | Release [v3.2.6](https://fluentbit.io/announcements/v3.2.6/) |
| 3.2.5-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.2.5 | amd64, arm64, arm/v7, s390x | Release [v3.2.5](https://fluentbit.io/announcements/v3.2.5/) |
| 3.2.4-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.2.4 | amd64, arm64, arm/v7, s390x | Release [v3.2.4](https://fluentbit.io/announcements/v3.2.4/) |
| 3.2.3-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.2.3 | amd64, arm64, arm/v7, s390x | Release [v3.2.3](https://fluentbit.io/announcements/v3.2.3/) |
| 3.2.2-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.2.2 | amd64, arm64, arm/v7, s390x | Release [v3.2.2](https://fluentbit.io/announcements/v3.2.2/) |
| 3.2.1-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.2.1 | amd64, arm64, arm/v7, s390x | Release [v3.2.1](https://fluentbit.io/announcements/v3.2.1/) |
| 3.1.10-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.1.10 | amd64, arm64, arm/v7, s390x | Release [v3.1.10](https://fluentbit.io/announcements/v3.1.10/) |
| 3.1.9-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.1.9 | amd64, arm64, arm/v7, s390x | Release [v3.1.9](https://fluentbit.io/announcements/v3.1.9/) |
| 3.1.8-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.1.8 | amd64, arm64, arm/v7, s390x | Release [v3.1.8](https://fluentbit.io/announcements/v3.1.8/) |
| 3.1.7-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.1.7 | amd64, arm64, arm/v7, s390x | Release [v3.1.7](https://fluentbit.io/announcements/v3.1.7/) |
| 3.1.6-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.1.6 | amd64, arm64, arm/v7, s390x | Release [v3.1.6](https://fluentbit.io/announcements/v3.1.6/) |
| 3.1.5-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.1.5 | amd64, arm64, arm/v7, s390x | Release [v3.1.5](https://fluentbit.io/announcements/v3.1.5/) |
| 3.1.4-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.1.4 | amd64, arm64, arm/v7, s390x | Release [v3.1.4](https://fluentbit.io/announcements/v3.1.4/) |
| 3.1.3-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.1.3 | amd64, arm64, arm/v7, s390x | Release [v3.1.3](https://fluentbit.io/announcements/v3.1.3/) |
| 3.1.2-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.1.2 | amd64, arm64, arm/v7, s390x | Release [v3.1.2](https://fluentbit.io/announcements/v3.1.2/) |
| 3.1.1-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.1.1 | amd64, arm64, arm/v7, s390x | Release [v3.1.1](https://fluentbit.io/announcements/v3.1.1/) |
| 3.1.0-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.1.0 | amd64, arm64, arm/v7, s390x | Release [v3.1.0](https://fluentbit.io/announcements/v3.1.0/) |
| 3.0.7-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.0.7 | amd64, arm64, arm/v7, s390x | Release [v3.0.7](https://fluentbit.io/announcements/v3.0.7/) |
| 3.0.6-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.0.6 | amd64, arm64, arm/v7, s390x | Release [v3.0.6](https://fluentbit.io/announcements/v3.0.6/) |
| 3.0.5-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.0.5 | amd64, arm64, arm/v7, s390x | Release [v3.0.5](https://fluentbit.io/announcements/v3.0.5/) |
| 3.0.4-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.0.4 | amd64, arm64, arm/v7, s390x | Release [v3.0.4](https://fluentbit.io/announcements/v3.0.4/) |
| 3.0.3-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.0.3 | amd64, arm64, arm/v7, s390x | Release [v3.0.3](https://fluentbit.io/announcements/v3.0.3/) |
| 3.0.2-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.0.2 | amd64, arm64, arm/v7, s390x | Release [v3.0.2](https://fluentbit.io/announcements/v3.0.2/) |
| 3.0.1-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.0.1 | amd64, arm64, arm/v7, s390x | Release [v3.0.1](https://fluentbit.io/announcements/v3.0.1/) |
| 3.0.0-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 3.0.0 | amd64, arm64, arm/v7, s390x | Release [v3.0.0](https://fluentbit.io/announcements/v3.0.0/) |
| 2.2.2-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 2.2.2 | amd64, arm64, arm/v7, s390x | Release [v2.2.2](https://fluentbit.io/announcements/v2.2.2/) |
| 2.2.1-debug | amd64, arm64, arm/v7, s390x | Debug images |
| 2.2.1 | amd64, arm64, arm/v7, s390x | Release [v2.2.1](https://fluentbit.io/announcements/v2.2.1/) |
| 2.2.0-debug | amd64, arm64, arm/v7 | Debug images |
| 2.2.0 | amd64, arm64, arm/v7 | Release [v2.2.0](https://fluentbit.io/announcements/v2.2.0/) |
| 2.1.10-debug | amd64, arm64, arm/v7 | Debug images |
| 2.1.10 | amd64, arm64, arm/v7 | Release [v2.1.10](https://fluentbit.io/announcements/v2.1.10/) |
| 2.1.9-debug | amd64, arm64, arm/v7 | Debug images |
| 2.1.9 | amd64, arm64, arm/v7 | Release [v2.1.9](https://fluentbit.io/announcements/v2.1.9/) |
| 2.1.8-debug | amd64, arm64, arm/v7 | Debug images |
| 2.1.8 | amd64, arm64, arm/v7 | Release [v2.1.8](https://fluentbit.io/announcements/v2.1.8/) |
| 2.1.7-debug | amd64, arm64, arm/v7 | Debug images |
| 2.1.7 | amd64, arm64, arm/v7 | Release [v2.1.7](https://fluentbit.io/announcements/v2.1.7/) |
| 2.1.6-debug | amd64, arm64, arm/v7 | Debug images |
| 2.1.6 | amd64, arm64, arm/v7 | Release [v2.1.6](https://fluentbit.io/announcements/v2.1.6/) |
| 2.1.5 | amd64, arm64, arm/v7 | Release [v2.1.5](https://fluentbit.io/announcements/v2.1.5/) |
| 2.1.5-debug | amd64, arm64, arm/v7 | Debug images |
| 2.1.3 | amd64, arm64, arm/v7 | Release [v2.1.3](https://fluentbit.io/announcements/v2.1.3/) |
| 2.1.3-debug | amd64, arm64, arm/v7 | Debug images |
| 2.1.2 | amd64, arm64, arm/v7 | Release [v2.1.2](https://fluentbit.io/announcements/v2.1.2/) |
| 2.1.2-debug | amd64, arm64, arm/v7 | Debug images |
| 2.1.1 | amd64, arm64, arm/v7 | Release [v2.1.1](https://fluentbit.io/announcements/v2.1.1/) |
| 2.1.1-debug | amd64, arm64, arm/v7 | v2.1.x releases (production + debug) |
| 2.1.0 | amd64, arm64, arm/v7 | Release [v2.1.0](https://fluentbit.io/announcements/v2.1.0/) |
| 2.1.0-debug | amd64, arm64, arm/v7 | v2.1.x releases (production + debug) |
| 2.0.11 | amd64, arm64, arm/v7 | Release [v2.0.11](https://fluentbit.io/announcements/v2.0.11/) |
| 2.0.11-debug | amd64, arm64, arm/v7 | v2.0.x releases (production + debug) |
| 2.0.10 | amd64, arm64, arm/v7 | Release [v2.0.10](https://fluentbit.io/announcements/v2.0.10/) |
| 2.0.10-debug | amd64, arm64, arm/v7 | v2.0.x releases (production + debug) |
| 2.0.9 | amd64, arm64, arm/v7 | Release [v2.0.9](https://fluentbit.io/announcements/v2.0.9/) |
| 2.0.9-debug | amd64, arm64, arm/v7 | v2.0.x releases (production + debug) |
| 2.0.8 | amd64, arm64, arm/v7 | Release [v2.0.8](https://fluentbit.io/announcements/v2.0.8/) |
| 2.0.8-debug | amd64, arm64, arm/v7 | v2.0.x releases (production + debug) |
| 2.0.6 | amd64, arm64, arm/v7 | Release [v2.0.6](https://fluentbit.io/announcements/v2.0.6/) |
| 2.0.6-debug | amd64, arm64, arm/v7 | v2.0.x releases (production + debug) |
| 2.0.5 | amd64, arm64, arm/v7 | Release [v2.0.5](https://fluentbit.io/announcements/v2.0.5/) |
| 2.0.5-debug | amd64, arm64, arm/v7 | v2.0.x releases (production + debug) |
| 2.0.4 | amd64, arm64, arm/v7 | Release [v2.0.4](https://fluentbit.io/announcements/v2.0.4/) |
| 2.0.4-debug | amd64, arm64, arm/v7 | v2.0.x releases (production + debug) |
| 2.0.3 | amd64, arm64, arm/v7 | Release [v2.0.3](https://fluentbit.io/announcements/v2.0.3/) |
| 2.0.3-debug | amd64, arm64, arm/v7 | v2.0.x releases (production + debug) |
| 2.0.2 | amd64, arm64, arm/v7 | Release [v2.0.2](https://fluentbit.io/announcements/v2.0.2/) |
| 2.0.2-debug | amd64, arm64, arm/v7 | v2.0.x releases (production + debug) |
| 2.0.1 | amd64, arm64, arm/v7 | Release [v2.0.1](https://fluentbit.io/announcements/v2.0.1/) |
| 2.0.1-debug | amd64, arm64, arm/v7 | v2.0.x releases (production + debug) |
| 2.0.0 | amd64, arm64, arm/v7 | Release [v2.0.0](https://fluentbit.io/announcements/v2.0.0/) |
| 2.0.0-debug | amd64, arm64, arm/v7 | v2.0.x releases (production + debug) |
| 1.9.9 | amd64, arm64, arm/v7 | Release [v1.9.9](https://fluentbit.io/announcements/v1.9.9/) |
| 1.9.9-debug | amd64, arm64, arm/v7 | v1.9.x releases (production + debug) |
| 1.9.8 | amd64, arm64, arm/v7 | Release [v1.9.8](https://fluentbit.io/announcements/v1.9.8/) |
| 1.9.8-debug | amd64, arm64, arm/v7 | v1.9.x releases (production + debug) |
| 1.9.7 | amd64, arm64, arm/v7 | Release [v1.9.7](https://fluentbit.io/announcements/v1.9.7/) |
| 1.9.7-debug | amd64, arm64, arm/v7 | v1.9.x releases (production + debug) |
| 1.9.6 | amd64, arm64, arm/v7 | Release [v1.9.6](https://fluentbit.io/announcements/v1.9.6/) |
| 1.9.6-debug | amd64, arm64, arm/v7 | v1.9.x releases (production + debug) |
| 1.9.5 | amd64, arm64, arm/v7 | Release [v1.9.5](https://fluentbit.io/announcements/v1.9.5/) |
| 1.9.5-debug | amd64, arm64, arm/v7 | v1.9.x releases (production + debug) |
| 1.9.4 | amd64, arm64, arm/v7 | Release [v1.9.4](https://fluentbit.io/announcements/v1.9.4/) |
| 1.9.4-debug | amd64, arm64, arm/v7 | v1.9.x releases (production + debug) |
| 1.9.3 | amd64, arm64, arm/v7 | Release [v1.9.3](https://fluentbit.io/announcements/v1.9.3/) |
| 1.9.3-debug | amd64, arm64, arm/v7 | v1.9.x releases (production + debug) |
| 1.9.2 | amd64, arm64, arm/v7 | Release [v1.9.2](https://fluentbit.io/announcements/v1.9.2/) |
| 1.9.2-debug | amd64, arm64, arm/v7 | v1.9.x releases (production + debug) |
| 1.9.1 | amd64, arm64, arm/v7 | Release [v1.9.1](https://fluentbit.io/announcements/v1.9.1/) |
| 1.9.1-debug | amd64, arm64, arm/v7 | v1.9.x releases (production + debug) |
| 1.9.0 | amd64, arm64, arm/v7 | Release [v1.9.0](https://fluentbit.io/announcements/v1.9.0/) |
| 1.9.0-debug | amd64, arm64, arm/v7 | v1.9.x releases (production + debug) |
It's strongly suggested that you always use the latest image of Fluent Bit.
Container images for Windows Server 2019 and Windows Server 2022 are provided for v2.0.6 and later. These can be found as tags on the same Docker Hub registry.
## Multi-architecture images
Fluent Bit production stable images are based on [Distroless](https://github.com/GoogleContainerTools/distroless). Focusing on security, these images contain only the Fluent Bit binary and minimal system libraries and basic configuration.
Debug images are available for all architectures (for 1.9.0 and later), and contain a full Debian shell and package manager that can be used to troubleshoot or for testing purposes.
From a deployment perspective, there's no need to specify an architecture. The container client tool that pulls the image gets the proper layer for the running architecture.
## Verify signed container images
Version 1.9 and 2.0 container images are signed using Cosign/Sigstore. Verify these signatures using `cosign` ([install guide](https://docs.sigstore.dev/quickstart/quickstart-cosign/)):
```shell
$ cosign verify --key "https://packages.fluentbit.io/fluentbit-cosign.pub" fluent/fluent-bit:2.0.6
Verification for index.docker.io/fluent/fluent-bit:2.0.6 --
The following checks were performed on each of these signatures:
- The cosign claims were validated
- The signatures were verified against the specified public key
[{"critical":{"identity":{"docker-reference":"index.docker.io/fluent/fluent-bit"},"image":{"docker-manifest-digest":"sha256:c740f90b07f42823d4ecf4d5e168f32ffb4b8bcd87bc41df8f5e3d14e8272903"},"type":"cosign container image signature"},"optional":{"release":"2.0.6","repo":"fluent/fluent-bit","workflow":"Release from staging"}}]
```
Replace `cosign` with the binary installed if it has a different name (for example, `cosign-linux-amd64`).
Keyless signing is also provided but is still experimental:
```shell
COSIGN_EXPERIMENTAL=1 cosign verify fluent/fluent-bit:2.0.6
```
`COSIGN_EXPERIMENTAL=1` is used to allow verification of images signed in keyless mode. To learn more about keyless signing, see the [Sigstore keyless signature](https://docs.sigstore.dev/cosign/signing/overview/) documentation.
## Get started
1. Download the last stable image from 2.0 series:
```shell
docker pull cr.fluentbit.io/fluent/fluent-bit:2.0
```
2. After the image is in place, run the following test which makes Fluent Bit measure CPU usage by the container:
```shell
docker run -ti cr.fluentbit.io/fluent/fluent-bit:2.0 \
-i cpu -o stdout -f 1
```
That command lets Fluent Bit measure CPU usage every second and flushes the results to the standard output. For example:
```shell
[2019/10/01 12:29:02] [ info] [engine] started
[0] cpu.0: [1504290543.000487750, {"cpu_p"=>0.750000, "user_p"=>0.250000, "system_p"=>0.500000, "cpu0.p_cpu"=>0.000000, "cpu0.p_user"=>0.000000, "cpu0.p_system"=>0.000000, "cpu1.p_cpu"=>1.000000, "cpu1.p_user"=>0.000000, "cpu1.p_system"=>1.000000, "cpu2.p_cpu"=>1.000000, "cpu2.p_user"=>1.000000, "cpu2.p_system"=>0.000000, "cpu3.p_cpu"=>0.000000, "cpu3.p_user"=>0.000000, "cpu3.p_system"=>0.000000}]
```
## FAQ
### Why there is no Fluent Bit Docker image based on Alpine Linux?
Alpine Linux uses Musl C library instead of Glibc. Musl isn't fully compatible with Glibc, which generated many issues in the following areas when used with Fluent Bit:
* Memory Allocator: To run properly in high-load environments, Fluent Bit uses Jemalloc as a default memory allocator which reduces fragmentation and provides better performance. Jemalloc can't run smoothly with Musl and requires extra work.
* Alpine Linux Musl functions bootstrap have a compatibility issue when loading Golang shared libraries. This causes problems when trying to load Golang output plugins in Fluent Bit.
* Alpine Linux Musl Time format parser doesn't support Glibc extensions.
* The Fluent Bit maintainers' preference for base images are Distroless and Debian for security and maintenance reasons.
### Why use distroless containers?
The reasons for using distroless are well covered in
[Why should I use Distroless images?](https://github.com/GoogleContainerTools/distroless#why-should-i-use-distroless-images).
* Include only what you need, reduce the attack surface available.
* Reduces size and improves performance.
* Reduces false positives on scans (and reduces resources required for scanning).
* Reduces supply chain security requirements to only what you need.
* Helps prevent unauthorised processes or users interacting with the container.
* Less need to harden the container (and container runtime, K8s, and so on).
* Faster CI/CD processes.
With any choice, there are downsides:
* No shell or package manager to update or add things.
* Generally, dynamic updating is a bad idea in containers as the time it's done affects the outcome: two containers started at different times using the same base image can perform differently or get different dependencies.
* A better approach is to rebuild a new image version. You can do this with Distroless, but it's harder and requires multistage builds or similar to provide the new dependencies.
* Debugging can be harder.
* More specifically you need applications set up to properly expose information for debugging rather than rely on traditional debug approaches of connecting to processes or dumping memory. This can be an upfront cost versus a runtime cost but does shift left in the development process so hopefully is a reduction overall.
* Assumption that Distroless is secure: nothing is secure and there are still exploits so it doesn't remove the need for securing your system.
* Sometimes you need to use a common base image, such as with audits, security, health, and so on.
Using `exec` to access a container will potentially impact resource limits.
For debugging, debug containers are available now in K8S:
* This can be a significantly different container from the one you want to investigate, with lots of extra tools or even a different base.
* No resource limits applied to this container, which can be good or bad.
* Runs in pod namespaces. It's another container that can access everything the others can.
* Might need architecture of the pod to share volumes or other information.
* Requires more recent versions of K8S and the container runtime plus RBAC allowing it.
# Kubernetes
Kubernetes Production Grade Log Processor
[Fluent Bit](https://fluentbit.io) is a lightweight and extensible log processor with full support for Kubernetes:
* Process Kubernetes containers logs from the file system or Systemd/Journald.
* Enrich logs with Kubernetes Metadata.
* Centralize your logs in third party storage services like Elasticsearch, InfluxDB, HTTP, and so on.
## Concepts
Before getting started it's important to understand how Fluent Bit will be deployed. Kubernetes manages a cluster of nodes. The Fluent Bit log agent tool needs to run on every node to collect logs from every pod. Fluent Bit is deployed as a DaemonSet, which is a pod that runs on every node of the cluster.
When Fluent Bit runs, it reads, parses, and filters the logs of every pod. In addition, Fluent Bit adds metadata to each entry using the [Kubernetes](/manual/data-pipeline/filters/kubernetes) filter plugin.
The Kubernetes filter plugin talks to the Kubernetes API Server to retrieve relevant information such as the `pod_id`, `labels`, and `annotations`. Other fields, such as `pod_name`, `container_id`, and `container_name`, are retrieved locally from the log file names. All of this is handled automatically, and no intervention is required from a configuration aspect.
## Installation
Fluent Bit should be deployed as a DaemonSet, so it will be available on every node of your Kubernetes cluster.
The recommended way to deploy Fluent Bit for Kubernetes is with the official [Helm Chart](https://github.com/fluent/helm-charts).
### OpenShift
If you are using Red Hat OpenShift you must set up Security Context Constraints (SCC) using the relevant option in the helm chart.
### Installing with Helm chart
[Helm](https://helm.sh) is a package manager for Kubernetes and lets you deploy application packages into your running cluster. Fluent Bit is distributed using a Helm chart found in the [Fluent Helm Charts repository](https://github.com/fluent/helm-charts).
Use the following command to add the Fluent Helm charts repository
```shell
helm repo add fluent https://fluent.github.io/helm-charts
```
To validate that the repository was added, run `helm search repo fluent` to ensure the charts were added. The default chart can then be installed by running the following command:
```shell
helm upgrade --install fluent-bit fluent/fluent-bit
```
### Default values
The default chart values include configuration to read container logs. With Docker parsing, Systemd logs apply Kubernetes metadata enrichment, and output to an Elasticsearch cluster. You can modify the [included values file](https://github.com/fluent/helm-charts/blob/main/charts/fluent-bit/values.yaml) to specify additional outputs, health checks, monitoring endpoints, or other configuration options.
## Details
The default configuration of Fluent Bit ensures the following:
* Consume all containers logs from the running node and parse them with either the `docker` or `cri` multi-line parser.
* Persist how far it got into each file it's tailing so if a pod is restarted it picks up from where it left off.
* The Kubernetes filter adds Kubernetes metadata, specifically `labels` and `annotations`. The filter only contacts the API Server when it can't find the cached information, otherwise it uses the cache.
* The default backend in the configuration is Elasticsearch set by the [Elasticsearch Output Plugin](/manual/data-pipeline/outputs/elasticsearch). It uses the Logstash format to ingest the logs. If you need a different `Index` and `Type`, refer to the plugin option and update as needed.
* There is an option called `Retry_Limit`, which is set to `False`. If Fluent Bit can't flush the records to Elasticsearch, it will retry indefinitely until it succeeds.
## Windows deployment
Fluent Bit v1.5.0 and later supports deployment to Windows pods.
### Log files overview
When deploying Fluent Bit to Kubernetes, there are three log files that you need to pay attention to.
* `C:\k\kubelet.err.log`
This is the error log file from kubelet daemon running on host. Retain this file for future troubleshooting, including debugging deployment failures.
* `C:\var\log\containers\__-.log`
This is the main log file you need to watch. Configure Fluent Bit to follow this file. It's a symlink to the Docker log file in `C:\ProgramData\`, with some additional metadata on the file's name.
* `C:\ProgramData\Docker\containers\\.log`
This is the log file produced by Docker. Normally you don't directly read from this file, but you need to make sure that this file is visible from Fluent Bit.
Typically, your deployment YAML contains the following volume configuration.
```yaml
spec:
containers:
- name: fluent-bit
image: my-repo/fluent-bit:1.8.4
volumeMounts:
- mountPath: C:\k
name: k
- mountPath: C:\var\log
name: varlog
- mountPath: C:\ProgramData
name: progdata
volumes:
- name: k
hostPath:
path: C:\k
- name: varlog
hostPath:
path: C:\var\log
- name: progdata
hostPath:
path: C:\ProgramData
```
### Configure Fluent Bit
Assuming the basic volume configuration described previously, you can apply one of the following configurations to start logging:
{% tabs %}
{% tab title="fluent-bit.yaml" %}
```yaml
parsers:
- name: docker
format: json
time_key: time
time_format: '%Y-%m-%dT%H:%M:%S.%L'
time_keep: true
pipeline:
inputs:
- name: tail
tag: kube.*
path: 'C:\\var\\log\\containers\\*.log'
parser: docker
db: 'C:\\fluent-bit\\tail_docker.db'
mem_buf_limit: 7MB
refresh_interval: 10
- name: tail
tag: kube.error
path: 'C:\\k\\kubelet.err.log'
db: 'C:\\fluent-bit\\tail_kubelet.db'
filters:
- name: kubernetes
match: kube.*
kube_url: 'https://kubernetes.default.svc.cluster.local:443'
outputs:
- name: stdout
match: '*'
```
{% endtab %}
{% tab title="fluent-bit.conf" %}
```
fluent-bit.conf: |
[SERVICE]
Parsers_File C:\\fluent-bit\\parsers.conf
[INPUT]
Name tail
Tag kube.*
Path C:\\var\\log\\containers\\*.log
Parser docker
DB C:\\fluent-bit\\tail_docker.db
Mem_Buf_Limit 7MB
Refresh_Interval 10
[INPUT]
Name tail
Tag kubelet.err
Path C:\\k\\kubelet.err.log
DB C:\\fluent-bit\\tail_kubelet.db
[FILTER]
Name kubernetes
Match kube.*
Kube_URL https://kubernetes.default.svc.cluster.local:443
[OUTPUT]
Name stdout
Match *
parsers.conf: |
[PARSER]
Name docker
Format json
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L
Time_Keep On
```
{% endtab %}
{% endtabs %}
### Mitigate unstable network on Windows pods
Windows pods often lack working DNS immediately after boot ([#78479](https://github.com/kubernetes/kubernetes/issues/78479)). To mitigate this issue, `filter_kubernetes` provides a built-in mechanism to wait until the network starts up:
* `DNS_Retries`: Retries N times until the network start working (6)
* `DNS_Wait_Time`: Lookup interval between network status checks (30)
By default, Fluent Bit waits for three minutes (30 seconds x 6 times). If it's not enough for you, update the configuration as follows:
{% tabs %}
{% tab title="fluent-bit.yaml" %}
```yaml
filters:
- name: kubernetes
...
dns_retries: 10
dns_wait_time: 30
```
{% endtab %}
{% tab title="fluent-bit.conf" %}
```
[filter]
Name kubernetes
...
DNS_Retries 10
DNS_Wait_Time 30
```
{% endtab %}
{% endtabs %}
# macOS
Fluent Bit is compatible with the latest Apple macOS software for x86\_64 and Apple Silicon architectures.
## Installation packages
Installation packages can be found [in the Fluent Bit repository](https://packages.fluentbit.io/).
## Requirements
You must have [Homebrew](https://brew.sh/) installed in your system. If it isn't present, install it with the following command:
```shell
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
```
## Installing from Homebrew
The Fluent Bit package on Homebrew isn't officially supported, but should work for basic use cases and testing. It can be installed using:
```shell
brew install fluent-bit
```
## Compile from source
### Install build dependencies
Run the following brew command in your terminal to retrieve the dependencies:
```shell
brew install git cmake openssl bison libyaml
```
## Download and build the source
1. Download a copy of the Fluent Bit source code (upstream):
```shell
git clone https://github.com/fluent/fluent-bit
```
1. Go to the Fluent Bit directory.
```shell
cd fluent-bit
```
If you want to use a specific version, checkout to the proper tag. For example, to use `v4.0.4`, use the command:
```shell
git checkout v4.0.4
```
1. To prepare the build system, you must export certain environment variables so Fluent Bit CMake build rules can pick the right libraries:
```shell
export OPENSSL_ROOT_DIR=`brew --prefix openssl`
export PATH=`brew --prefix bison`/bin:$PATH
```
1. Change to the `build/` directory inside the Fluent Bit sources:
```shell
cd build/
```
1. Build Fluent Bit. This example indicates to the build system the location the final binaries and `config` files should be installed:
```shell
cmake -DFLB_DEV=on -DCMAKE_INSTALL_PREFIX=/opt/fluent-bit ../
make -j 16
```
1. Install Fluent Bit to the previously specified directory. Writing to this directory requires root privileges.
```shell
sudo make install
```
The binaries and configuration examples can be located at `/opt/fluent-bit/`.
## Create macOS installer from source
1. Clone the Fluent Bit source code (upstream):
```shell
git clone https://github.com/fluent/fluent-bit
```
1. Change to the Fluent Bit directory
```shell
cd fluent-bit
```
To use a specific version, checkout to the proper tag. For example, to use `v4.0.4` do:
```shell
git checkout v4.0.4
```
1. To prepare the build system, you must expose certain environment variables so Fluent Bit CMake build rules can pick the right libraries:
```shell
export OPENSSL_ROOT_DIR=`brew --prefix openssl`
export PATH=`brew --prefix bison`/bin:$PATH
```
1. Create the specific macOS SDK target. For example, to specify macOS Big Sur (11.3) SDK environment:
```shell
export MACOSX_DEPLOYMENT_TARGET=11.3
```
1. Change to the `build/` directory inside the Fluent Bit sources:
```shell
cd build/
```
1. Build the Fluent Bit macOS installer:
```shell
cmake -DCPACK_GENERATOR=productbuild -DCMAKE_INSTALL_PREFIX=/opt/fluent-bit ../
make -j 16
cpack -G productbuild
```
The macOS installer will be generated as:
```
CPack: Create package using productbuild
CPack: Install projects
CPack: - Run preinstall target for: fluent-bit
CPack: - Install project: fluent-bit []
CPack: - Install component: binary
CPack: - Install component: library
CPack: - Install component: headers
CPack: - Install component: headers-extra
CPack: Create package
CPack: - Building component package: /Users/fluent-bit-builder/GitHub/fluent-bit/build/_CPack_Packages/Darwin/productbuild//Users/fluent-bit-builder/GitHub/fluent-bit/build/fluent-bit-1.9.2-apple/Contents/Packages/fluent-bit-1.9.2-apple-binary.pkg
CPack: - Building component package: /Users/fluent-bit-builder/GitHub/fluent-bit/build/_CPack_Packages/Darwin/productbuild//Users/fluent-bit-builder/GitHub/fluent-bit/build/fluent-bit-1.9.2-apple/Contents/Packages/fluent-bit-1.9.2-apple-headers.pkg
CPack: - Building component package: /Users/fluent-bit-builder/GitHub/fluent-bit/build/_CPack_Packages/Darwin/productbuild//Users/fluent-bit-builder/GitHub/fluent-bit/build/fluent-bit-1.9.2-apple/Contents/Packages/fluent-bit-1.9.2-apple-headers-extra.pkg
CPack: - Building component package: /Users/fluent-bit-builder/GitHub/fluent-bit/build/_CPack_Packages/Darwin/productbuild//Users/fluent-bit-builder/GitHub/fluent-bit/build/fluent-bit-1.9.2-apple/Contents/Packages/fluent-bit-1.9.2-apple-library.pkg
CPack: - package: /Users/fluent-bit-builder/GitHub/fluent-bit/build/fluent-bit-1.9.2-apple.pkg generated.
```
Finally, the `fluent-bit--(intel or apple)`.pkg will be generated.
The created installer will put binaries at `/opt/fluent-bit/`.
## Running Fluent Bit
To make the access path easier to Fluent Bit binary, extend the `PATH` variable:
```shell
export PATH=/opt/fluent-bit/bin:$PATH
```
To test, try Fluent Bit by generating a test message using the [Dummy input plugin](/manual/data-pipeline/inputs/dummy) which prints to the standard output interface every one second:
```shell
fluent-bit -i dummy -o stdout -f 1
```
You will see an output similar to this:
```
...
[0] dummy.0: [1644362033.676766000, {"message"=>"dummy"}]
[0] dummy.0: [1644362034.676914000, {"message"=>"dummy"}]
```
To halt the process, press `ctrl-c` in the terminal.
# Windows
Fluent Bit is distributed as the `fluent-bit` package for Windows and as a [Windows container on Docker Hub](/manual/installation/downloads/docker). Fluent Bit provides two Windows installers: a `ZIP` archive and an `EXE` installer.
Not all plugins are supported on Windows. The [CMake configuration](https://github.com/fluent/fluent-bit/blob/master/cmake/windows-setup.cmake) shows the default set of supported plugins.
## Configuration
Provide a valid Windows configuration with the installation.
The following configuration is an example:
{% tabs %}
{% tab title="fluent-bit.yaml" %}
```yaml
service:
flush: 5
daemon: off
log_level: info
parsers_file: parsers.yaml
plugins_file: plugins.yaml
http_server: off
http_listen: 0.0.0.0
http_port: 2020
storage.metrics: on
pipeline:
inputs:
- name: winlog
channels: Setup,Windows Powershell
interval_sec: 1
outputs:
- name: stdout
match: '*'
```
{% endtab %}
{% tab title="fluent-bit.conf" %}
```
[SERVICE]
# Flush
# =====
# set an interval of seconds before to flush records to a destination
flush 5
# Daemon
# ======
# instruct Fluent Bit to run in foreground or background mode.
daemon Off
# Log_Level
# =========
# Set the verbosity level of the service, values can be:
#
# - error
# - warning
# - info
# - debug
# - trace
#
# by default 'info' is set, that means it includes 'error' and 'warning'.
log_level info
# Parsers File
# ============
# specify an optional 'Parsers' configuration file
parsers_file parsers.conf
# Plugins File
# ============
# specify an optional 'Plugins' configuration file to load external plugins.
plugins_file plugins.conf
# HTTP Server
# ===========
# Enable/Disable the built-in HTTP Server for metrics
http_server Off
http_listen 0.0.0.0
http_port 2020
# Storage
# =======
# Fluent Bit can use memory and filesystem buffering based mechanisms
#
# - https://docs.fluentbit.io/manual/administration/buffering-and-storage
#
# storage metrics
# ---------------
# publish storage pipeline metrics in '/api/v1/storage'. The metrics are
# exported only if the 'http_server' option is enabled.
#
storage.metrics on
[INPUT]
Name winlog
Channels Setup,Windows PowerShell
Interval_Sec 1
[OUTPUT]
name stdout
match *
```
{% endtab %}
{% endtabs %}
## Migration to Fluent Bit
For version 1.9 and later, `td-agent-bit` is a deprecated package and was removed after 1.9.9. The correct package name to use now is `fluent-bit`.
## Installation packages
The latest stable version is 5.0.8.
| INSTALLERS | SHA256 CHECKSUMS |
| ------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| [`fluent-bit-5.0.8-win32.exe`](https://packages.fluentbit.io/windows/fluent-bit-5.0.8-win32.exe) | [5384bd1dad1970d112e93905b4cc39497840e665919372198c22c42567cf642f](https://packages.fluentbit.io/windows/fluent-bit-5.0.8-win32.exe.sha256) |
| [`fluent-bit-5.0.8-win32.zip`](https://packages.fluentbit.io/windows/fluent-bit-5.0.8-win32.zip) | [203eafec85e89f77e12fa60e340a97c21bf4845b5156b9dcfd962abdd2111654](https://packages.fluentbit.io/windows/fluent-bit-5.0.8-win32.zip.sha256) |
| [`fluent-bit-5.0.8-win64.exe`](https://packages.fluentbit.io/windows/fluent-bit-5.0.8-win64.exe) | [f8183135337696cb3f8581ecbca13fe2b502621e86b90c964657eeb9715a9fd9](https://packages.fluentbit.io/windows/fluent-bit-5.0.8-win64.exe.sha256) |
| [`fluent-bit-5.0.8-win64.zip`](https://packages.fluentbit.io/windows/fluent-bit-5.0.8-win64.zip) | [b5155a6fd2edecfc2b36a0efea20d2d9c7872aada98177fb0b271a738fbfba87](https://packages.fluentbit.io/windows/fluent-bit-5.0.8-win64.zip.sha256) |
| [`fluent-bit-5.0.8-winarm64.exe`](https://packages.fluentbit.io/windows/fluent-bit-5.0.8-winarm64.exe) | [b13dc0686ddace93f85acedb76924229baf318e4f12d1757fcc858b7db01c19d](https://packages.fluentbit.io/windows/fluent-bit-5.0.8-winarm64.exe.sha256) |
| [`fluent-bit-5.0.8-winarm64.zip`](https://packages.fluentbit.io/windows/fluent-bit-5.0.8-winarm64.zip) | [53aea0b1c05f543c6bb73e074d7a68efe51f3ab45833ef9979a5217b11320baa](https://packages.fluentbit.io/windows/fluent-bit-5.0.8-winarm64.zip.sha256) |
These are now using the Github Actions built versions. Legacy AppVeyor builds are still available (AMD 32/64 only) at releases.fluentbit.io but are deprecated.
MSI installers are also available:
* [`fluent-bit-5.0.8-win32.msi`](https://packages.fluentbit.io/windows/fluent-bit-5.0.8-win32.msi)
* [`fluent-bit-5.0.8-win64.msi`](https://packages.fluentbit.io/windows/fluent-bit-5.0.8-win64.msi)
* [`fluent-bit-5.0.8-winarm64.msi`](https://packages.fluentbit.io/windows/fluent-bit-5.0.8-winarm64.msi)
To check the integrity, use the `Get-FileHash` cmdlet for PowerShell.
```shell
Get-FileHash fluent-bit-5.0.8-win32.exe
```
## Installing from a ZIP archive
1. Download a ZIP archive. Choose the suitable installers for your 32-bit or 64-bit environments.
2. Expand the ZIP archive. You can do this by clicking **Extract All** in Explorer or `Expand-Archive` in PowerShell.
```shell
Expand-Archive fluent-bit-5.0.8-win64.zip
```
The ZIP package contains the following set of files.
```
fluent-bit
├── bin
│ ├── fluent-bit.dll
│ └── fluent-bit.exe
│ └── fluent-bit.pdb
├── conf
│ ├── fluent-bit.conf
│ ├── parsers.conf
│ └── plugins.conf
└── include
│ ├── flb_api.h
│ ├── ...
│ └── flb_worker.h
└── fluent-bit.h
```
3. Launch `cmd.exe` or PowerShell on your machine, and execute `fluent-bit.exe`:
```shell
fluent-bit.exe -i dummy -o stdout
```
The following output indicates Fluent Bit is running:
```shell
fluent-bit.exe -i dummy -o stdout
...
[0] dummy.0: [1561684385.443823800, {"message"=>"dummy"}]
[1] dummy.0: [1561684386.428399000, {"message"=>"dummy"}]
[2] dummy.0: [1561684387.443641900, {"message"=>"dummy"}]
[3] dummy.0: [1561684388.441405800, {"message"=>"dummy"}]
```
To halt the process, press `Control+C` in the terminal.
## Installing from the executable installer
1. Download an `EXE` installer for the appropriate 32-bit or 64-bit build.
2. Double-click the `EXE` installer you've downloaded. The installation wizard starts.
3. Click **Next** and finish the installation. By default, Fluent Bit is installed in `C:\Program Files\fluent-bit\`.
```shell
& "C:\Program Files\fluent-bit\bin\fluent-bit.exe" -i dummy -o stdout
```
### Installer options
The Windows installer is built by [`CPack` using NSIS](https://cmake.org/cmake/help/latest/cpack_gen/nsis.html) and supports the [default NSIS options](https://nsis.sourceforge.io/Docs/Chapter3.html#3.2.1) for silent installation and install directory.
To silently install to `C:\fluent-bit` directory here is an example:
```shell
/S /D=C:\fluent-bit
```
The uninstaller also supports a silent uninstall using the same `/S` flag. This can be used for provisioning with automation like Ansible, Puppet, and so on.
## Windows service support
Windows services are equivalent to daemons in Unix (long-running background processes). For v1.5.0 and later, Fluent Bit has native support for Windows services.
For example, you have the following installation layout:
```
C:\fluent-bit\
├── conf
│ ├── fluent-bit.conf
│ └── parsers.conf
│ └── plugins.conf
└── bin
├── fluent-bit.dll
└── fluent-bit.exe
└── fluent-bit.pdb
```
To register Fluent Bit as a Windows service, execute the following command on at a command prompt. A single space is required after `binpath=`.
```shell
sc.exe create fluent-bit binpath= "\fluent-bit\bin\fluent-bit.exe -c \fluent-bit\conf\fluent-bit.conf"
```
Fluent Bit can be started and managed as a normal Windows service.
```shell
sc.exe start fluent-bit
sc.exe query fluent-bit
SERVICE_NAME: fluent-bit
TYPE : 10 WIN32_OWN_PROCESS
STATE : 4 Running
...
```
To halt the Fluent Bit service, use the `stop` command.
```shell
sc.exe stop fluent-bit
```
To start Fluent Bit automatically on boot, execute the following:
```shell
sc.exe config fluent-bit start= auto
```
## FAQs
### Can you manage Fluent Bit service using PowerShell?
Instead of `sc.exe`, PowerShell can be used to manage Windows services.
Create a Fluent Bit service:
```shell
New-Service fluent-bit -BinaryPathName "`"C:\Program Files\fluent-bit\bin\fluent-bit.exe`" -c `"C:\Program Files\fluent-bit\conf\fluent-bit.conf`"" -StartupType Automatic -Description "This service runs Fluent Bit, a log collector that enables real-time processing and delivery of log data to centralized logging systems."
```
Start the service:
```shell
Start-Service fluent-bit
```
Query the service status:
```shell
get-Service fluent-bit | format-list
Name : fluent-bit
DisplayName : fluent-bit
Status : Running
DependentServices : {}
ServicesDependedOn : {}
CanPauseAndContinue : False
CanShutdown : False
CanStop : True
ServiceType : Win32OwnProcess
```
Stop the service:
```shell
Stop-Service fluent-bit
```
Remove the service (requires PowerShell 6.0 or later)
```shell
Remove-Service fluent-bit
```
## Compile from source
If you need to create a custom executable, use the following procedure to compile Fluent Bit by yourself.
### Preparation
1. Install Microsoft Visual C++ to compile Fluent Bit. You can install the minimum toolkit using the following command:
```shell
wget -o vs.exe https://aka.ms/vs/16/release/vs_buildtools.exe
start vs.exe
```
1. Choose `C++ Build Tools` and `C++ CMake tools for Windows` and wait until the process finishes.
2. Install flex and bison. One way to install them on Windows is to use [`winflexbison`](https://github.com/lexxmark/winflexbison).
```shell
wget -o winflexbison.zip https://github.com/lexxmark/winflexbison/releases/download/v2.5.22/win_flex_bison-2.5.22.zip
Expand-Archive winflexbison.zip -Destination C:\WinFlexBison
cp -Path C:\WinFlexBison\win_bison.exe C:\WinFlexBison\bison.exe
cp -Path C:\WinFlexBison\win_flex.exe C:\WinFlexBison\flex.exe
```
3. Add the path `C:\WinFlexBison` to your systems environment variable `Path`. [Here's how to do that](https://www.architectryan.com/2018/03/17/add-to-the-path-on-windows-10/).
4. Install OpenSSL binaries, at least the library files and headers.
5. Install [Git](https://git-scm.com/download/win) to pull the source code from the repository.
```shell
wget -o git.exe https://github.com/git-for-windows/git/releases/download/v2.28.0.windows.1/Git-2.28.0-64-bit.exe
start git.exe
```
### Compilation
1. Open the **Start menu** on Windows and type `command Prompt for VS`. From the result list, select the one that corresponds to your target system ( `x86` or `x64`).
2. Verify the installed OpenSSL library files match the selected target. You can examine the library files by using the `dumpbin` command with the `/headers` option.
3. Clone the source code of Fluent Bit.
```shell
git clone https://github.com/fluent/fluent-bit
cd fluent-bit/build
```
4. Compile the source code.
```shell
cmake .. -G "NMake Makefiles"
cmake --build .
```
Now you should be able to run Fluent Bit:
```shell
fluent-bit.exe -i dummy -o stdout
```
### Packaging
To create a ZIP package, call `cpack` as follows:
```shell
cpack -G ZIP
```
# What's new in Fluent Bit v5.0
Fluent Bit `v5.0` adds new inputs and processors, expands authentication and TLS options, and standardizes configuration for HTTP-based plugins. It also delivers an important round of performance and scalability work, especially for pipelines that ingest logs, metrics, and traces through HTTP-based protocols. This page gives a quick user-focused overview of the main changes since Fluent Bit `v4.2`.
For migration-impacting changes, see [Upgrade notes](/manual/installation/upgrade-notes).
## Performance and scalability
### Unified processing and delivery model
Fluent Bit `v5.0` continues the move toward a more unified runtime for logs, metrics, and traces. In practice, this means the same core engine improvements benefit more of the pipeline, instead of individual signal paths evolving separately.
For end users, the result is a more consistent behavior across telemetry types and a better base for high-throughput pipelines that mix logs, metrics, and traces in the same deployment.
### Refactored HTTP stack
One of the most important `v5.0` changes is the refactoring of the HTTP listener stack used by several input plugins. Fluent Bit now uses a shared HTTP server implementation across the major HTTP-based receivers instead of maintaining separate code paths.
This work improves:
* concurrency through shared listener worker support
* consistency of request handling across HTTP-based inputs
* buffer enforcement and connection handling
* maintainability, which reduces drift between plugin implementations
The biggest user-facing beneficiaries are:
* [HTTP input](/manual/data-pipeline/inputs/http)
* [Splunk input](/manual/data-pipeline/inputs/splunk)
* [Elasticsearch input](/manual/data-pipeline/inputs/elasticsearch)
* [OpenTelemetry input](/manual/data-pipeline/inputs/opentelemetry)
* [Prometheus remote write input](/manual/data-pipeline/inputs/prometheus-remote-write)
If you run large HTTP or OTLP ingestion workloads, `v5.0` is not only a feature release. It is also a meaningful runtime improvement.
## Configuration and operations
### Shared HTTP listener settings
HTTP-based inputs now use a shared listener configuration model. The preferred setting names are:
* `http_server.http2`
* `http_server.buffer_chunk_size`
* `http_server.buffer_max_size`
* `http_server.max_connections`
* `http_server.workers`
* `http_server.ingress_queue_event_limit`
* `http_server.ingress_queue_byte_limit`
Legacy aliases such as `http2`, `buffer_chunk_size`, and `buffer_max_size` still work, but new configurations should use the `http_server.*` names.
Affected plugin families include:
* [HTTP input](/manual/data-pipeline/inputs/http)
* [Splunk input](/manual/data-pipeline/inputs/splunk)
* [Elasticsearch input](/manual/data-pipeline/inputs/elasticsearch)
* [OpenTelemetry input](/manual/data-pipeline/inputs/opentelemetry)
* [Prometheus remote write input](/manual/data-pipeline/inputs/prometheus-remote-write)
### Mutual TLS for inputs
Input plugins that support TLS can now require client certificate verification with `tls.verify_client_cert`. This makes it easier to run mutual TLS (`mTLS`) directly on Fluent Bit listeners.
See [TLS](/manual/administration/transport-security).
### JSON health endpoint in API v2
The built-in HTTP server exposes `/api/v2/health`, which returns health status as JSON and uses the HTTP status code to indicate healthy (`200`) or unhealthy (`500`) state.
See [Monitoring](/manual/administration/monitoring).
## Inputs
### New `fluentbit_logs` input
The [Fluent Bit logs input](/manual/data-pipeline/inputs/fluentbit-logs) routes Fluent Bit internal logs back into the pipeline as structured records. This lets you forward agent diagnostics to any supported destination.
### HTTP input remote address capture
The [HTTP input](/manual/data-pipeline/inputs/http) adds:
* `add_remote_addr`
* `remote_addr_key`
These settings let you attach the client address from `X-Forwarded-For` to each ingested record.
### `OAuth 2.0` bearer token validation on HTTP-based inputs
HTTP-based receivers can validate incoming bearer tokens with:
* `oauth2.validate`
* `oauth2.issuer`
* `oauth2.jwks_url`
* `oauth2.allowed_audience`
* `oauth2.allowed_clients`
* `oauth2.jwks_refresh_interval`
This is available on the relevant input plugins, including [HTTP](/manual/data-pipeline/inputs/http) and [OpenTelemetry](/manual/data-pipeline/inputs/opentelemetry).
### OpenTelemetry input improvements
The [OpenTelemetry input](/manual/data-pipeline/inputs/opentelemetry) in `v5.0` expands user-visible behavior with:
* shared HTTP listener worker support
* `OAuth 2.0` bearer token validation
* stable JSON metrics ingestion over `OTLP/HTTP`
* improved JSON trace validation and error reporting
### Kubernetes events state database controls
The [Kubernetes events input](/manual/data-pipeline/inputs/kubernetes-events) documents additional SQLite controls:
* `db.journal_mode`
* `db.locking`
These settings help tune event cursor persistence and database access behavior.
## Processors
### New cumulative-to-delta processor
The [cumulative to delta processor](/manual/data-pipeline/processors/cumulative-to-delta) converts cumulative monotonic metrics to delta values, which is useful when scraping Prometheus-style metrics but exporting to backends that expect deltas.
### New topological data analysis processor
The [topological data analysis processor](/manual/data-pipeline/processors/tda) adds a metrics processor for topology-based analysis workflows.
### Sampling processor updates
The [sampling processor](/manual/data-pipeline/processors/sampling) adds `legacy_reconcile` for tail sampling, which helps compare the optimized reconciler with the previous behavior when validating upgrades.
## Outputs
### HTTP output `OAuth 2.0` client credentials
The [HTTP output](/manual/data-pipeline/outputs/http) now supports built-in `OAuth 2.0` client credentials with:
* `basic`
* `post`
* `private_key_jwt`
You can configure token acquisition directly in Fluent Bit with the `oauth2.*` settings.
### More compression options for cloud outputs
Several outputs gained additional compression support in the `v4.2` to `v5.0` range:
* [Amazon Kinesis Data Streams](/manual/data-pipeline/outputs/kinesis): `gzip`, `zstd`, `snappy`
* [Amazon Kinesis Data Firehose](/manual/data-pipeline/outputs/firehose): `snappy` added alongside existing codecs
* [Amazon S3](/manual/data-pipeline/outputs/s3): `snappy` added alongside existing codecs
* [Azure Blob](/manual/data-pipeline/outputs/azure_blob): `zstd` support for transfer compression
## Monitoring changes
### `fluentbit_hot_reloaded_times` is now a counter
The `fluentbit_hot_reloaded_times` metric changed from a gauge to a counter, which makes it safe to use with PromQL functions such as `rate()` and `increase()`.
### New output backpressure visibility
`v5.0` adds output backpressure duration metrics so you can observe time spent waiting because of downstream pressure.
See [Monitoring](/manual/administration/monitoring).
# Upgrade notes
The following article covers the relevant compatibility changes for users upgrading from previous Fluent Bit versions.
For more details about changes on each release, refer to the [Official Release Notes](https://fluentbit.io/announcements/).
Release notes will be prepared in advance of a Git tag for a release. An official release should provide both a tag and a release note together to allow users to verify and understand the release contents.
The tag drives the binary release process. Release binaries (containers and packages) will appear after a tag and its associated release note. This lets users to expect the new release binary to appear and allow/deny/update it as appropriate in their infrastructure.
## Fluent Bit v5.0
### `hot_reloaded_times` metric type change
The internal metric `fluentbit_hot_reloaded_times` has changed from a gauge to a counter. The previous gauge registration caused incorrect results when using PromQL functions like `rate()` and `increase()`, which expect counters.
If you have Prometheus dashboards or alerting rules that reference `fluentbit_hot_reloaded_times`, update them to use counter-appropriate PromQL functions (for example, `rate()` or `increase()` instead of gauge-specific functions like `delta()`).
### Shared HTTP listener settings for HTTP-based inputs
The HTTP-based input plugins now use a shared HTTP listener configuration model. In Fluent Bit `v5.0`, the canonical setting names are:
* `http_server.http2`
* `http_server.buffer_chunk_size`
* `http_server.buffer_max_size`
* `http_server.max_connections`
* `http_server.workers`
* `http_server.ingress_queue_event_limit`
* `http_server.ingress_queue_byte_limit`
Legacy per-plugin names such as `http2`, `buffer_chunk_size`, and `buffer_max_size` are still accepted as compatibility aliases, but new configurations should use the `http_server.*` names.
If you tune `http`, `splunk`, `elasticsearch`, `opentelemetry`, or `prometheus_remote_write` inputs, review those sections and migrate to the shared naming so future upgrades are clearer.
### Mutual TLS for input plugins
Input plugins that support TLS now also support `tls.verify_client_cert`. Enable this option to require and validate the client certificate presented by the sender.
If you terminate TLS directly in Fluent Bit and need mutual TLS (`mTLS`), add `tls.verify_client_cert on` together with the usual `tls.crt_file` and `tls.key_file` settings.
### New internal logs input
Fluent Bit `v5.0` adds the `fluentbit_logs` input plugin, which mirrors the Fluent Bit internal log stream back into the data pipeline as structured log records.
Use this input if you want to forward Fluent Bit diagnostics to another destination, filter them, or store them alongside the rest of your telemetry.
### Emitter backpressure with filesystem storage
The internal emitter plugin, used by filters such as `rewrite_tag`, now automatically enables `storage.pause_on_chunks_overlimit` when filesystem storage is in use and that option hasn't been explicitly configured.
Previously, the emitter could accumulate chunks beyond the `storage.max_chunks_up` limit. Pipelines that use `rewrite_tag` or other emitter-backed filters with filesystem storage will now pause when the configured storage limit is reached.
If you rely on the previous unlimited accumulation behavior, explicitly set `storage.pause_on_chunks_overlimit off` on the relevant input. Otherwise, review your `storage.max_chunks_up` value to ensure it's tuned for your expected throughput.
### More `OAuth 2.0` coverage
Fluent Bit `v5.0` expands `OAuth 2.0` support in both directions:
* HTTP-based inputs can validate incoming bearer tokens using `oauth2.validate`, `oauth2.issuer`, and `oauth2.jwks_url`.
* The HTTP output can acquire access tokens with `oauth2.enable` and supports `basic`, `post`, and `private_key_jwt` client authentication.
If you previously handled authentication outside Fluent Bit for these cases, review the plugin pages for the new built-in options.
### Forward output: `retain_metadata_in_forward_mode` default changed to `true`
Starting in Fluent Bit v5.0.4, the `forward` output plugin's `retain_metadata_in_forward_mode` option defaults to `true` (previously `false`). When `true`, Fluent Bit embeds event metadata into the Forward protocol payload using the extended MessagePack format.
Fluentd receivers don't understand this format and will reject events with a warning similar to:
```
[input1] skip invalid event: host="..." tag="..." time=[..., {}] record={...}
```
If you send data from Fluent Bit to a Fluentd aggregator using the `forward` output, explicitly set `retain_metadata_in_forward_mode false` to restore the previous behavior:
{% tabs %}
{% tab title="YAML" %}
```yaml
pipeline:
outputs:
- name: forward
match: "*"
host: fluentd-host
port: 24224
retain_metadata_in_forward_mode: false
```
{% endtab %}
{% tab title="Classic" %}
```
[OUTPUT]
Name forward
Match *
Host fluentd-host
Port 24224
Retain_Metadata_In_Forward_Mode false
```
{% endtab %}
{% endtabs %}
For more details, see [GitHub issue #11877](https://github.com/fluent/fluent-bit/issues/11877).
For a broader overview of user-visible additions in this release, see [What's new in Fluent Bit v5.0](/manual/installation/whats-new-in-fluent-bit-v5.0).
## Fluent Bit v4.2
### Vivo exporter output plugin
The HTTP endpoint paths exposed by the Vivo exporter output plugin have changed. All endpoints now follow an `/api/v1/` prefix:
| Signal | Endpoint |
| ---------------- | -------------------------- |
| Logs | `/api/v1/logs` |
| Metrics | `/api/v1/metrics` |
| Traces | `/api/v1/traces` |
| Internal metrics | `/api/v1/internal/metrics` |
If you have tooling or dashboards that query the Vivo exporter HTTP endpoints directly, update the endpoint paths accordingly.
## Fluent Bit v4.0
### Package support for older Linux distributions
Official binary packages are no longer produced for the following Linux distributions:
* Ubuntu 16.04 (`Xenial`)
* Ubuntu 18.04 (`Bionic`)
* Ubuntu 20.04 (`Focal`)
Users on these platforms should upgrade their OS or build Fluent Bit from source.
### Kafka plugin support on older platforms
The Kafka input and output plugins are disabled in official packages for CentOS 7 and Amazon Linux 2 (ARM64). This is due to a Kafka library (`librdkafka`) update that requires a newer glibc version than these platforms provide.
Users who need Kafka support on these platforms must build Fluent Bit from source against a compatible version of `librdkafka`.
## Fluent Bit v3.0
### HTTP/2 enabled by default for HTTP-based input plugins
The following input plugins now have HTTP/2 support enabled by default (`http2 true`):
* `opentelemetry`
* `splunk`
* `elasticsearch`
* `http`
These plugins transparently support both HTTP/1.1 and HTTP/2 connections. If your clients don't support HTTP/2, or if you have a reverse proxy or load balancer that doesn't handle HTTP/2 correctly, add `http2 off` to the affected input plugin configuration section.
## Fluent Bit v2.0
### TLS library
mbedTLS is no longer supported as a TLS backend. All TLS connections now use OpenSSL. If you compile Fluent Bit from source and previously linked against mbedTLS, you must now link against OpenSSL. Official binary packages already use OpenSSL.
## Fluent Bit v1.9.9
The `td-agent-bit` package is no longer provided after this release. Users should switch to the `fluent-bit` package.
## Fluent Bit v1.6
If you are migrating from previous version of Fluent Bit, review the following important changes:
### Tail input plugin
By default, the tail input plugin follows a file from the end after the service starts, instead of reading it from the beginning. Every file found when the plugin starts is followed from it last position. New files discovered at runtime or when files rotate are read from the beginning.
To keep the old behavior, set the option `read_from_head` to `true`.
### Stackdriver output plugin
The `project_id` of [resource](https://docs.cloud.google.com/logging/docs/reference/v2/rest/v2/MonitoredResource) in [LogEntry](https://docs.cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry) sent to Google Cloud Logging would be set to the `project_id` rather than the project number. To learn the difference between Project ID and project number, see [Creating and managing projects](https://docs.cloud.google.com/resource-manager/docs/creating-managing-projects#before_you_begin).
If you have existing queries based on the resource's `project_id,` update your query accordingly.
## Fluent Bit v1.5
The migration from v1.4 to v1.5 is pretty straightforward.
* The `keepalive` configuration mode has been renamed to `net.keepalive`. Now, all Network I/O keepalive is enabled by default. To learn more about this and other associated configuration properties read the [Networking Administration](https://docs.fluentbit.io/manual/administration/networking#tcp-keepalive) section. - If you use the Elasticsearch output plugin, the default value of `type` [changed from `flb_type` to `_doc`](https://github.com/fluent/fluent-bit/commit/04ed3d8104ca8a2f491453777ae6e38e5377817e#diff-c9ae115d3acaceac5efb949edbb21196). Many versions of Elasticsearch tolerate this, but Elasticsearch v5.6 through v6.1 require a `type` without a leading underscore. See the [Elasticsearch output plugin documentation FAQ entry](https://docs.fluentbit.io/manual/pipeline/outputs/elasticsearch#faq-underscore) for more.
## Fluent Bit v1.4
If you are migrating from Fluent Bit v1.3, there are no breaking changes.
## Fluent Bit v1.3
If you are migrating from Fluent Bit v1.2 to v1.3, there are no breaking changes. If you are upgrading from an older version, review the following incremental changes:
## Fluent Bit v1.2
### Docker, JSON, parsers, and decoders
Fluent Bit v1.2 fixed many issues associated with JSON encoding and decoding.
For example, when parsing Docker logs, it's no longer necessary to use decoders. The new Docker parser looks like this:
```
[PARSER]
Name docker
Format json
Time_Key time
Time_Format %Y-%m-%dT%H:%M:%S.%L
Time_Keep On
```
### Kubernetes filter
Fluent Bit made improvements to Kubernetes Filter handling of string-encoded `log` messages. If the `Merge_Log` option is enabled, it will try to handle the log content as a JSON map, if so, it will add the keys to the root map.
In addition, fixes and improvements were made to the `Merge_Log_Key` option. If a merge log succeed, all new keys will be packaged under the key specified by this option. A suggested configuration is as follows:
```
[FILTER]
Name Kubernetes
Match kube.*
Kube_Tag_Prefix kube.var.log.containers.
Merge_Log On
Merge_Log_Key log_processed
```
As an example, if the original log content is the following map:
```json
{"key1": "val1", "key2": "val2"}
```
the final record will be composed as follows:
```json
{"log": "{\"key1\": \"val1\", \"key2\": \"val2\"}", "log_processed": { "key1": "val1", "key2": "val2" } }
```
## Fluent Bit v1.1
If you are upgrading from Fluent Bit 1.0.x or earlier, review the following relevant changes when switching to Fluent Bit v1.1 or later series:
### Kubernetes filter
Fluent Bit introduced a new configuration property called `Kube_Tag_Prefix` to help Tag prefix resolution and address an unexpected behavior in previous versions.
During the `1.0.x` release cycle, a commit in the Tail input plugin changed the default behavior on how the Tag was composed when using the wildcard for expansion generating breaking compatibility with other services. Consider the following configuration example:
```
[INPUT]
Name tail
Path /var/log/containers/*.log
Tag kube.*
```
The expected behavior is that Tag will be expanded to:
`text kube.var.log.containers.apache.log`
The change introduced in the 1.0 series switched from absolute path to the base filename only:
`text kube.apache.log`
THe Fluent Bit v1.1 release restored the default behavior and now the Tag is composed using the absolute path of the monitored file.
Having absolute path in the Tag is relevant for routing and flexible configuration where it also helps to keep compatibility with Fluentd behavior.
This behavior switch in Tail input plugin affects how Filter Kubernetes operates. When the filter is used it needs to perform local metadata lookup that comes from the file names when using Tail as a source. With the new `Kube_Tag_Prefix` option you can specify the prefix used in the Tail input plugin. For the previous configuration example the new configuration will look like:
```
[INPUT]
Name tail
Path /var/log/containers/*.log
Tag kube.*
[FILTER]
Name kubernetes
Match *
Kube_Tag_Prefix kube.var.log.containers.
```
The proper value for `Kube_Tag_Prefix` must be composed by Tag prefix set in Tail input plugin plus the converted monitored directory replacing slashes with dots.
# Configure Fluent Bit
Fluent Bit uses configuration files to store information about your specified [inputs](/manual/data-pipeline/inputs), [outputs](/manual/data-pipeline/outputs), [filters](/manual/data-pipeline/filters), and more. You can write these configuration files in one of these formats:
* [YAML configuration files](/manual/administration/configuring-fluent-bit/yaml) are the standard configuration format as of Fluent Bit v3.2. They use the `.yaml` file extension.
* [Classic configuration files](/manual/administration/configuring-fluent-bit/classic-mode) will be deprecated at the end of 2026. They use the `.conf` file extension.
## Unit sizes
Some configuration settings in Fluent Bit use standardized unit sizes to define data and storage limits. For example, the `buffer_chunk_size` and `buffer_max_size` parameters for the [Tail](/manual/data-pipeline/inputs/tail) input plugin use unit sizes.
The following table describes the unit sizes you can use and what they mean.
| Suffix | Description | Example |
| -------------------- | --------------------------------------------------------------------------------------------------------------------- | --------------------------------- |
| *none* | **Bytes**: If you specify an integer without a unit size, Fluent Bit interprets that value as a bytes representation. | `32000` means 32,000 bytes. |
| `k`, `kb`, `K`, `KB` | **Kilobytes**: A unit of memory equal to 1,000 bytes. | `32k` means 32,000 bytes. |
| `m`, `mb`, `M`, `MB` | **Megabytes**: A unit of memory equal to 1,000,000 bytes. | `32m` means 32,000,000 bytes. |
| `g`, `gb`, `G`, `GB` | **Gigabytes**: A unit of memory equal to 1,000,000,000 bytes. | `32g` means 32,000,000,000 bytes. |
## Command line interface
Fluent Bit exposes most of its configuration features through the command line interface. Use the `-h` or `--help` flag to see a list of available options.
```shell
# Podman container tooling.
podman run -rm -ti fluent/fluent-bit --help
# Docker container tooling.
docker run --rm -it fluent/fluent-bit --help
```
### Validate configuration with `--dry-run`
Use the `--dry-run` flag to validate a configuration file without starting Fluent Bit:
```shell
fluent-bit --dry-run -c /path/to/fluent-bit.yaml
```
A successful validation prints `configuration test is successful` and exits with code `0`. If validation fails, Fluent Bit exits with a non-zero code and prints the errors to stderr.
As of Fluent Bit 4.2, `--dry-run` performs full property validation in addition to syntax checking. Prior to 4.2, unknown or misspelled plugin property names would only surface as errors at runtime; `--dry-run` now catches them during validation. For example, a configuration with an unknown property on a `dummy` input produces:
```
[error] [config] dummy: unknown configuration property 'invalid_property_that_does_not_exist'.
```
# YAML configuration files
In Fluent Bit v3.2 and later, YAML configuration files support all of the settings and features that [classic configuration files](/manual/administration/configuring-fluent-bit/classic-mode) support, plus additional features that classic configuration files don't support, like processors.
YAML configuration files support the following top-level sections:
* `env`: Configures [environment variables](/manual/administration/configuring-fluent-bit/yaml/environment-variables-section).
* `includes`: Specifies additional YAML configuration files to [include as part of a parent file](/manual/administration/configuring-fluent-bit/yaml/includes-section).
* `service`: Configures global properties of the Fluent Bit [service](/manual/administration/configuring-fluent-bit/yaml/service-section).
* `pipeline`: Configures active [`inputs`, `filters`, and `outputs`](/manual/administration/configuring-fluent-bit/yaml/pipeline-section).
* `parsers`: Defines [custom parsers](/manual/administration/configuring-fluent-bit/yaml/parsers-section).
* `multiline_parsers`: Defines [custom multiline parsers](/manual/administration/configuring-fluent-bit/yaml/multiline-parsers-section).
* `plugins`: Defines paths for [custom plugins](/manual/administration/configuring-fluent-bit/yaml/plugins-section).
* `upstream_servers`: Defines [nodes](/manual/administration/configuring-fluent-bit/yaml/upstream-servers-section) for output plugins.
{% hint style="info" %}
YAML configuration is used in the smoke tests for containers. An always-correct up-to-date example is here: .
{% endhint %}
# Environment variables
The `env` section of YAML configuration files lets you define environment variables. These variables can then be used to dynamically replace values throughout your configuration using the `${VARIABLE_NAME}` syntax. The `env` section supports two forms: a map form for static key-value pairs and an extended list form for advanced use cases like file-backed secrets with automatic refresh.
Values set in the `env` section are case-sensitive. However, as a best practice, Fluent Bit recommends using uppercase names for environment variables.
## Map form
The map form defines variables as key-value pairs. The following example defines two variables, `FLUSH_INTERVAL` and `STDOUT_FMT`, which can be accessed in the configuration using `${FLUSH_INTERVAL}` and `${STDOUT_FMT}`:
```yaml
env:
FLUSH_INTERVAL: 1
STDOUT_FMT: 'json_lines'
service:
flush: ${FLUSH_INTERVAL}
log_level: info
pipeline:
inputs:
- name: random
outputs:
- name: stdout
match: '*'
format: ${STDOUT_FMT}
```
## Extended list form
The `env` section also accepts a YAML list of entries. Each entry is a mapping with the following properties:
| Property | Required | Description |
| ------------------ | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `name` | Yes | The name of the environment variable. |
| `value` | No | An explicit value for the variable. Mutually exclusive with `uri`. |
| `uri` | No | A URI pointing to the variable's value source. Currently supports `file://` URIs (for example, `file:///run/secrets/my_secret`). The file contents are read and used as the variable's value. |
| `refresh_interval` | No | How often (in seconds) to re-read the value from the `uri` source. Only meaningful when `uri` is set. Default is `0` (read once at startup). |
Either `value` or `uri` must be provided for each entry.
When `uri` and `refresh_interval` are both set, Fluent Bit re-reads the file at the specified interval. This enables dynamic configuration values, such as rotating secrets, without restarting Fluent Bit. If `refresh_interval` is `0` or omitted, the file is read only once at startup.
```yaml
env:
- name: DB_PASSWORD
uri: file:///run/secrets/db_password
refresh_interval: 60
- name: REGION
value: us-east-1
service:
flush: 1
log_level: info
pipeline:
inputs:
- name: random
outputs:
- name: stdout
match: '*'
```
In this example, `${DB_PASSWORD}` is read from the file `/run/secrets/db_password` and refreshed every 60 seconds. `${REGION}` is set to the static value `us-east-1`.
## Default values
{% hint style="info" %}
Minimum Fluent Bit version 5.0.6
{% endhint %}
Fluent Bit supports setting a default value for an environment variable if the variable isn't defined or is set to an empty string ('').
The syntax for a default value is `${VARIABLE_NAME:-DEFAULT_VALUE}`.
```yaml
service:
flush: ${FLUSH_INTERVAL:-5}
log_level: info
```
## Predefined variables
Fluent Bit supports the following predefined environment variables. You can reference these variables in configuration files without defining them in the `env` section.
| Name | Description |
| ------------- | ---------------------- |
| `${HOSTNAME}` | The system's hostname. |
## External variables
In addition to variables defined in the configuration file or the predefined ones, Fluent Bit can access system environment variables set in the user space. These external variables can be referenced in the configuration using the same `${VARIABLE_NAME}` pattern.
{% hint style="info" %}
Variables set in the `env` section can't be overridden by system environment variables.
{% endhint %}
For example, to set the `FLUSH_INTERVAL` system environment variable to `2` and use it in your configuration:
```shell
export FLUSH_INTERVAL=2
```
In the configuration file, you can then access this value as follows:
```yaml
service:
flush: ${FLUSH_INTERVAL}
log_level: info
pipeline:
inputs:
- name: random
outputs:
- name: stdout
match: '*'
format: json_lines
```
This approach lets you manage and override configuration values using environment variables, providing flexibility in various deployment environments.
# Includes
The `includes` section of YAML configuration files lets you specify additional YAML files to be merged into the current configuration. This lets you organize complex configurations into smaller, manageable files and include them as needed.
These files are identified as a list of filenames and can include relative or absolute paths. If a path isn't specified as absolute, it will be treated as relative to the file that includes it.
## Usage
The following example demonstrates how to include additional YAML files using relative path references. This is the file system path structure:
```
├── fluent-bit.yaml
├── inclusion-1.yaml
└── subdir
└── inclusion-2.yaml
```
{% hint style="info" %}
Environment variables aren't supported in includes section. The path for each file must be specified as a literal string.
{% endhint %}
You can reference these files in `fluent-bit.yaml` as follows:
```yaml
includes:
- inclusion-1.yaml
- subdir/inclusion-2.yaml
```
Ensure that the included files are formatted correctly and contain valid YAML configurations for seamless integration.
# Service
The `service` section of YAML configuration files defines global properties of the Fluent Bit service. The available configuration keys are:
| Key | Description | Default Value |
| --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- |
| `coro_stack_size` | Sets the coroutines stack size in bytes. The value must be greater than the page size of the running system. Setting the value too small (for example, `4096`) can cause coroutine threads to overrun the stack buffer. For best results, don't change this parameter from its default value. | `24576` |
| `daemon` | Specifies whether Fluent Bit should run as a daemon (background process). Possible values: `yes`, `no`, `on`, and `off`. Don't enable when using a Systemd-based unit, such as the one provided in Fluent Bit packages. | `off` |
| `dns.mode` | Sets the primary transport layer protocol used by the asynchronous DNS resolver. Can be overridden on a per-plugin basis. | `UDP` |
| `dns.prefer_ipv4` | If enabled, the DNS resolver prefers IPv4 results when resolving hostnames. Possible values: `off` or `on`. | `off` |
| `dns.prefer_ipv6` | If enabled, the DNS resolver prefers IPv6 results when resolving hostnames. Possible values: `off` or `on`. | `off` |
| `dns.resolver` | Sets the DNS resolver implementation. Possible values: `LEGACY`, `ASYNC`. | *none* |
| `enable_chunk_trace` | If enabled, activates chunk tracing for debugging purposes. Requires Fluent Bit to be built with the `FLB_HAVE_CHUNK_TRACE` option. Possible values: `off` or `on`. | `off` |
| `flush` | Sets the flush time in `seconds.nanoseconds`. The engine loop uses a flush timeout to define when to flush the records ingested by input plugins through the defined output plugins. | `1` |
| `grace` | Sets the grace time in `seconds` as an integer value. The engine loop uses a grace timeout to define the wait time on exit. | `5` |
| `hc_errors_count` | Sets the number of errors that must occur within the health check period before the health check endpoint reports an unhealthy status. | `5` |
| `hc_period` | Sets the health check evaluation period in seconds. | `60` |
| `hc_retry_failure_count` | Sets the number of retry failures that must occur within the health check period before the health check endpoint reports an unhealthy status. | `5` |
| `health_check` | If enabled, registers a health check endpoint on the built-in HTTP server. Requires `http_server` to be enabled. Possible values: `off` or `on`. | `off` |
| `hot_reload` | Enables [hot reloading](/manual/administration/hot-reload) of configuration with SIGHUP. | `on` |
| `hot_reload.ensure_thread_safety` | If enabled, ensures thread safety during configuration hot reload. Disabling this can reduce reload time but can cause instability. Possible values: `off` or `on`. | `on` |
| `hot_reload.timeout` | Sets a watchdog timeout in seconds for the hot reload process. If the reload doesn't complete within this time, Fluent Bit cancels the reload. A value of `0` disables the watchdog. | `0` |
| `http_listen` | Sets the listening interface for the HTTP Server when it's enabled. | `0.0.0.0` |
| `http_port` | Sets the TCP port for the HTTP server. | `2020` |
| `http_server` | Enables the built-in HTTP server. | `off` |
| `json.convert_nan_to_null` | If enabled, `NaN` is converted to `null` when Fluent Bit converts `msgpack` to JSON. | `false` |
| `json.escape_unicode` | Controls how Fluent Bit serializes non‑ASCII / multi‑byte Unicode characters in JSON strings. When enabled, Unicode characters are escaped as `\uXXXX` sequences (characters outside BMP become surrogate pairs). When disabled, Fluent Bit emits raw UTF‑8 bytes. | `true` |
| `log_file` | Absolute path for an optional log file. By default, all logs are redirected to the standard error interface (`stderr`). | *none* |
| `log_level` | Sets the logging verbosity level. Possible values: `off`, `error`, `warn`, `info`, `debug`, and `trace`. Values are cumulative. For example, if `debug` is set, it will include `error`, `warning`, `info`, and `debug`. The `trace` mode is only available if Fluent Bit was built with the `WITH_TRACE` option enabled. | `info` |
| `multiline_buffer_limit` | Sets the default buffer size limit for multiline parsers. This value must follow [unit size](/manual/administration/configuring-fluent-bit#unit-sizes) specifications. | `2MB` |
| `parsers_file` | Path for [standalone parsers configuration files](/manual/administration/configuring-fluent-bit/yaml/parsers-section#standalone-parsers-files). You can include one or more files. | *none* |
| `plugins_file` | Path for a `plugins` configuration file. This file specifies the paths to external plugins (.so files) that Fluent Bit can load at runtime. Plugins can also be declared directly in the [`plugins` section](/manual/administration/configuring-fluent-bit/yaml/plugins-section) of YAML configuration files. | *none* |
| `scheduler.base` | Sets the base of exponential backoff. | `5` |
| `scheduler.cap` | Sets a maximum retry time in seconds. | `2000` |
| `sp.convert_from_str_to_num` | If enabled, the stream processor converts strings that represent numbers to a numeric type. | `true` |
| `streams_file` | Path for the [stream processor](/manual/stream-processing/overview) configuration file. This file defines the rules and operations for stream processing in Fluent Bit. Stream processor configurations can also be defined directly in the `streams` section of YAML configuration files. | *none* |
| `windows.maxstdio` | If specified, adjusts the limit of `stdio`. Only provided for Windows. Values from `512` to `2048` are allowed. | `512` |
The `service` section only controls the built-in monitoring and control HTTP server. Plugin-specific HTTP listener settings such as `http_server.http2`, `http_server.buffer_max_size`, `http_server.buffer_chunk_size`, `http_server.max_connections`, `http_server.workers`, `http_server.ingress_queue_event_limit`, `http_server.ingress_queue_byte_limit`, and `http_server.idle_timeout` are configured on the relevant input plugin in the [`pipeline.inputs`](/manual/administration/configuring-fluent-bit/yaml/pipeline-section#shared-http-listener-settings-for-inputs) section.
## Storage configuration
The following storage-related keys can be set as children to the `storage` key:
| Key | Description | Default Value |
| ------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- |
| `storage.backlog.flush_on_shutdown` | If enabled, Fluent Bit attempts to flush all backlog filesystem chunks to their destination during the shutdown process. This can help ensure data delivery before Fluent Bit stops, but can also increase shutdown time. Possible values: `off` or `on`. | `off` |
| `storage.backlog.mem_limit` | Sets the memory limit used by the `storage_backlog` input plugin when promoting backlog chunks (filesystem chunks left over from a previous Fluent Bit run) back into memory so they can be flushed by output plugins. While the up chunks owned by `storage_backlog` consume less memory than this limit, Fluent Bit continues to promote additional backlog chunks. This setting doesn't cap memory use for other input plugins that use filesystem storage. | `5M` |
| `storage.checksum` | Enables data integrity check when writing and reading data from the filesystem. The storage layer uses the CRC32 algorithm. Possible values: `off` or `on`. | `off` |
| `storage.delete_irrecoverable_chunks` | If enabled, deletes irrecoverable chunks during runtime and at startup. Possible values: `off` or `on`. | `off` |
| `storage.inherit` | If enabled, input plugins that don't explicitly set `storage.type` will inherit the global `storage.type` value. Possible values: `off` or `on`. | `off` |
| `storage.keep.rejected` | If enabled, the [dead letter queue](/manual/administration/dead-letter-queue) stores failed chunks that can't be delivered. Possible values: `off` or `on`. | `off` |
| `storage.max_chunks_up` | Sets the number of chunks that can be `up` in memory for input plugins that use filesystem storage. | `128` |
| `storage.metrics` | If `http_server` option is enabled in the main `service` section, this option registers a new endpoint where internal metrics of the storage layer can be consumed. For more details, see [Monitoring](/manual/administration/monitoring). Possible values: `off` or `on`. | `off` |
| `storage.path` | Sets a location to store streams and chunks of data. If this parameter isn't set, input plugins can't use filesystem buffering. | *none* |
| `storage.rejected.limit` | Sets the maximum total size of the dead letter queue. Accepts size values such as `100M` or `1G`. When the limit is reached, new rejected chunks are skipped and a warning is logged. If not set, the DLQ size is unlimited. | *none* |
| `storage.rejected.path` | Sets the subdirectory name under `storage.path` for storing rejected chunks in the dead letter queue. | `rejected` |
| `storage.sync` | Configures the synchronization mode used to store data in the file system. Using `full` increases the reliability of the filesystem buffer and ensures that data is guaranteed to be synced to the filesystem even if Fluent Bit crashes. On Linux, `full` corresponds with the `MAP_SYNC` option for [memory mapped files](https://man7.org/linux/man-pages/man2/mmap.2.html). Possible values: `normal`, `full`. | `normal` |
| `storage.trim_files` | If enabled, Fluent Bit trims chunk files in the filesystem to reclaim disk space after data is flushed. Possible values: `off` or `on`. | `off` |
| `storage.type` | Sets the default storage type for input plugins. Used in conjunction with `storage.inherit` to apply this type to all inputs that don't explicitly set their own `storage.type`. Possible values: `memory`, `filesystem`, `memrb`. | *none* |
For storage and buffering details, see [Buffering](/manual/data-pipeline/buffering) and [Backpressure](/manual/administration/backpressure).
For scheduler and retry details, see [Scheduling and retries](/manual/administration/scheduling-and-retries#Scheduling-and-Retries).
## Configuration example
The following configuration example defines a `service` section with [hot reloading](/manual/administration/hot-reload) enabled and a pipeline with a `random` input and `stdout` output:
```yaml
service:
flush: 1
log_level: info
http_server: true
http_listen: 0.0.0.0
http_port: 2020
hot_reload: on
pipeline:
inputs:
- name: random
outputs:
- name: stdout
match: '*'
```
# Parsers
You can define customer [parsers](/manual/data-pipeline/parsers) in the `parsers` section of YAML configuration files.
{% hint style="info" %}
To define custom multiline parsers, use [the `multiline_parsers` section](/manual/administration/configuring-fluent-bit/yaml/multiline-parsers-section) of YAML configuration files.
{% endhint %}
## Syntax
To define custom parsers in the `parsers` section of a YAML configuration file, use the following syntax.
{% tabs %}
{% tab title="fluent-bit.yaml" %}
```yaml
parsers:
- name: custom_parser1
format: json
time_key: time
time_format: '%Y-%m-%dT%H:%M:%S.%L'
time_keep: on
- name: custom_parser2
format: regex
regex: '^\<(?[0-9]{1,5})\>1 (?