Fluent Bit is a Fast and Lightweight Logs and Metrics Processor and Forwarder for Linux, OSX, Windows and BSD family operating systems. It has been made with a strong focus on performance to allow the collection of events from different sources without complexity.

Features

• High Performance

• Data Parsing

  • Convert your unstructured messages using our parsers: JSON, Regex, LTSV and Logfmt

• Metrics Collection (Prometheus compatible)

• Reliability and Data Integrity

  • Backpressure Handling

  • Data Buffering in memory and file system

• Networking

  • Security: built-in TLS/SSL support

  • Asynchronous I/O

• Pluggable Architecture and Extensibility: Inputs, Filters and Outputs

  • More than 80 built-in plugins available

  • Extensibility

    • Write any input, filter or output plugin in C language

    • Bonus: write Filters in Lua or Output plugins in Golang

• Monitoring: expose internal metrics over HTTP in JSON and Prometheus format

• Stream Processing: Perform data selection and transformation using simple SQL queries

  • Create new streams of data using query results

  • Aggregation Windows

  • Data analysis and prediction: Timeseries forecasting

• Portable: runs on Linux, MacOS, Windows and BSD systems

Fluent Bit, Fluentd and CNCF

Fluent Bit is a CNCF sub-project under the umbrella of Fluentd, it's licensed under the terms of the Apache License v2.0. This project was originally created by Treasure Data and is currently a vendor neutral and community driven project.

Fluent Bit provides different Input Plugins to gather information from different sources, some of them just collect data from log files while others can gather metrics information from the operating system. There are many plugins for different needs.

When an input plugin is loaded, an internal instance is created. Every instance has its own and independent configuration. Configuration keys are often called properties.

Every input plugin has its own documentation section where it's specified how it can be used and what properties are available.

For more details, please refer to the Input Plugins section.

After the project was around for some time, it got some traction in the Embedded market but we also started getting requests for several features from the Cloud community like more inputs, filters, and outputs. Not so long after that, Fluent Bit becomes one of the preferred solutions to solve the logging challenges in Cloud environments.

• Licensed under the terms of Apache License v2.0

• Production Grade solutions: deployed thousands of times every single day, millions per month.

• Community driven projects

• Widely Adopted by the Industry: trusted by all major companies like AWS, Microsoft, Google Cloud and hundred of others.

The following table describes a comparison in different areas of the projects:

In production environments we want to have full control of the data we are collecting, filtering is an important feature that allows us to alter the data before delivering it to some destination.

Filtering is implemented through plugins, so each filter available could be used to match, exclude or enrich your logs with some specific metadata.

We support many filters, A common use case for filtering is Kubernetes deployments. Every Pod log needs to get the proper metadata associated

Very similar to the input plugins, Filters run in an instance context, which has its own independent configuration. Configuration keys are often called properties.