Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
​Fluent Bit is an open source and multi-platform log processor tool which aims to be a generic Swiss knife for logs processing and distribution.
Nowadays the number of sources of information in our environments is ever increasing. Handling data collection at scale is complex, and collecting and aggregating diverse data requires a specialized tool that can deal with:
Different sources of information
Different data formats
Data Reliability
Security
Flexible Routing
Multiple destinations
Fluent Bit has been designed with performance and low resources consumption in mind.
The Production Grade Ecosystem
Licensed under the terms of Apache License v2.0
Production Grade solutions: deployed thousands of times every single day, millions per month.
Community driven projects
Widely Adopted by the Industry: trusted by all major companies like AWS, Microsoft, Google Cloud and hundred of others.
The following table describes a comparison in different areas of the projects:
High Performance Logs Processor
High Performance
Data Parsing
Reliability and Data Integrity
Networking
Security: built-in TLS/SSL support
Asynchronous I/O
More than 50 built-in plugins available
Extensibility
Write any input, filter or output plugin in C language
Create new streams of data using query results
Aggregation Windows
Data analysis and prediction: Timeseries forecasting
Portable: runs on Linux, MacOS, Windows and BSD systems
Logging and data processing in general can be complex, and at scale a bit more, that's why was born. But now is more than a simple tool, it's a full ecosystem that contains SDKs for different languages and sub projects like .
On this page, we will describe the relationship between the and open source projects, as a summary we can say both are:
Hosted projects by the
Originally created by .
Both projects share a lot of similarities, is fully designed and built on top of the best ideas of architecture and general design. Choosing which one to use depends on the end-user needs.
Both and can work as Aggregators or Forwarders, they both can complement each other or use them as standalone solutions.
is a Fast and Lightweight Log Processor, Stream Processor and Forwarder for Linux, OSX, Windows and BSD family operating systems. It has been made with a strong focus on performance to allow the collection of events from different sources without complexity.
Convert your unstructured messages using our parsers: , , and
Handling
in memory and file system
Pluggable Architecture and : Inputs, Filters and Outputs
Bonus: write or
: expose internal metrics over HTTP in JSON and format
: Perform data selection and transformation using simple SQL queries
is a sub-component of the project ecosystem, it's licensed under the terms of the . This project was created by and is its current primary sponsor.
Nowadays Fluent Bit get contributions from several companies and individuals and same as , it's hosted as a subproject.
Strong Commitment to the Openness and Collaboration
Fluent Bit, including it core, plugins and tools are distributed under the terms of the Apache License v2.0:
Fluentd | Fluent Bit |
Scope | Containers / Servers | Embedded Linux / Containers / Servers |
Language | C & Ruby | C |
Memory | ~40MB | ~650KB |
Performance | High Performance | High Performance |
Dependencies | Built as a Ruby Gem, it requires a certain number of gems. | Zero dependencies, unless some special plugin requires them. |
Plugins | More than 1000 plugins available | Around 70 plugins available |
License |
The way to gather data from your sources
When an input plugin is loaded, an internal instance is created. Every instance has its own and independent configuration. Configuration keys are often called properties.
Every input plugin has its own documentation section where it's specified how it can be used and what properties are available.
Modify, Enrich or Drop your records
In production environments we want to have full control of the data we are collecting, filtering is an important feature that allows us to alter the data before delivering it to some destination.
Filtering is implemented through plugins, so each filter available could be used to match, exclude or enrich your logs with some specific metadata.
We support many filters, A common use case for filtering is Kubernetes deployments. Every Pod log needs to get the proper metadata associated
Very similar to the input plugins, Filters run in an instance context, which has its own independent configuration. Configuration keys are often called properties.
provides different Input Plugins to gather information from different sources, some of them just collect data from log files while others can gather metrics information from the operating system. There are many plugins for different needs.
For more details, please refer to the section.
For more details about the Filters available and their usage, please refer to the section.