Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Loading...
Fluent Bit is an open source and multi-platform log forwarder tool which aims to be a generic Swiss knife for log collection and distribution.
We, Treasure Data, as a Big Data company, provide an analytics infrastructure in the Cloud where we provide an end-to-end solution to collect, store and do analytics over the data. Fluent Bit is an integral part of this pipeline where it solves the log collection needs.
Being an open source project, it has been widely adopted to solve logging needs in Cloud Native environments where Docker and Kubernetes are key components; Fluent Bit is a natural fit.
Data collection and log forwarding is hard.
Nowadays the number of sources of information in our environments is ever increasing. Handling data collection at scale is complex, and collecting and aggregating diverse data requires a specialized tool that can deal with:
Different sources of information.
Different data formats.
Multiple destinations.
Fluent Bit was born to address the need for a high performance and optimized tool that can collect data from any input source, unify that data and deliver it to multiple destinations.
Fluent Bit uses very low CPU and Memory consumption, it's compatible with most of x86, x86_64, AArch32 and AArch64 based platforms. In order to build it you need the following components in your system:
Compiler: GCC or clang
CMake
Flex (only if Stream Processor is enabled)
Bison (only if Stream Processor is enabled)
There are not other dependencies besides libc and pthreads in the most basic mode. For certain features that depends on third party components, those are included in the main source code repository.
Fluent Bit, including it core, plugins and tools are distributed under the terms of the Apache License v2.0:
The following section will guide you to the step to download, build and install Fluent Bit from sources and specific instructions for the installation of binaries that we already distribute for Debian/Ubuntu/Redhat/CentOS and Raspberry Pi.
If you find some problem on a certain step, don't hesitate to report the problem on our bug tracker:
Data collection matters and nowadays the scenarios from where the information can come from are very variable. For hence to be more flexible in certain markets needs, we may need different options. On this page, we will describe the relationship between the Fluentd and Fluent Bit open source projects.
Fluentd and Fluent Bit projects are both created and sponsored by Treasure Data and they aim to solve the collection, processing, and delivery of Logs.
Both projects share a lot of similarities, Fluent Bit is fully based on the design and experience of Fluentd architecture and general design. Choosing which one to use depends on the final needs, from an architecture perspective we can consider:
Fluentd is a log collector, processor, and aggregator.
Fluent Bit is a log collector and processor (it doesn't have strong aggregation features like Fluentd).
The following table describes a comparison in different areas of the projects:
Consider Fluentd mainly as an Aggregator and Fluent Bit as a Log Forwarder, we can see both projects complement each other providing a full reliable solution.
Fluent Bit container images are available on Docker Hub ready for production usage. Our stable images are based in Distroless focusing on security containing just the Fluent Bit binary, minimal system libraries and basic configuration.
Optionally, we provide debug images which contains Busybox that can be used to troubleshoot or testing purposes.
The following table describe the tags are available on fluent/fluent-bit repository:
It's strongly suggested that you always use the latest image of Fluent Bit.
Download the last stable image from 1.0 series:
Once the image is in place, now run the following (useless) test which makes Fluent Bit meassure CPU usage by the container:
That command will let Fluent Bit measure CPU usage every second and flush the results to the standard output, e.g:
Alpine Linux uses Musl C library instead of Glibc. Musl is not fully compatible with Glibc which generated many issues in the following areas when used with Fluent Bit:
Memory Allocator: to run Fluent Bit properly in high-load environments, we use Jemalloc as a default memory allocator which reduce fragmentation and provides better performance for our needs. Jemalloc cannot run smoothly with Musl and requires extra work.
Alpine Linux Musl functions bootstrap have a compatibility issue when loading Golang shared libraries, this generate problems when trying to load Golang output plugins in Fluent Bit.
Alpine Linux Musl Time format parser does not support Glibc extensions
Maintainers preference in terms of base image due to security and maintenance reasons are Distroless and Debian.
Fluent Bit is a Fast and Lightweight Log Processor and Forwarder for Linux, OSX and BSD family operating systems. It has been made with a strong focus on performance to allow the collection of events from different sources without complexity.
Fluent Bit is part of the Fluentd project ecosystem, it's licensed under the terms of the Apache License v2.0. This project is made and sponsored by Treasure Data.