External libraries

External libraries embedded in Fluent Bit

Fluent Bit embeds several external libraries in its /lib directory. These libraries are included to minimize external dependencies and maximize cross-platform compatibility.

Core fluent libraries

These libraries are developed and maintained by the Fluent Bit team:

Library	Purpose	License
CFL	Common data structures (strings, lists, key-value pairs, arrays, variants, time utilities, hashing) used across Fluent Bit and related projects.	Apache 2.0
Chunk I/O	Manages chunks of data in the file system for buffering. Handles memory-mapped I/O with optional CRC32 checksums for data integrity. Powers filesystem buffering for Fluent Bit.	Apache 2.0
CMetrics	Creates and manages metrics (counters, gauges, histograms, summaries) with label support. Based on Go Prometheus Client API design.	Apache 2.0
CTraces	Creates and manages trace contexts with encoding/decoding support for OpenTelemetry and other formats.	Apache 2.0
CProfiles	Creates and manages profiling data based on the OpenTelemetry Profiles schema.	Apache 2.0
Fluent OTel Proto	Provides C interfaces for OpenTelemetry protocol buffer definitions (common, resource, trace, logs, metrics).	Apache 2.0
flb_libco	Fork of libco for cooperative multithreading (coroutines). Includes ARMv8 fixes and macOS support.	Internet Systems Consortium

HTTP and networking

Library	Purpose	License
Monkey	Fast, lightweight HTTP server providing the embedded web server HTTP endpoints (health checks, metrics, and so on) for Fluent Bit.	Apache 2.0
c-ares	Modern asynchronous DNS resolver library. Enables non-blocking DNS queries for network operations.	Massachusetts Institute of Technology (MIT)
nghttp2	HTTP/2 C library implementing the framing layer. Enables HTTP/2 client and server capabilities.	Massachusetts Institute of Technology (MIT)

Data serialization and parsing

Library	Purpose	License
msgpack-c	MessagePack serialization library. Used for internal data format for records in Fluent Bit.	Boost 1.0
Jansson	JSON encoding, decoding, and manipulation with full Unicode support.	Massachusetts Institute of Technology (MIT)
yyjson	High-performance JSON library capable of processing gigabytes per second. Used for fast JSON operations.	Massachusetts Institute of Technology (MIT)
jsmn	Minimalistic JSON parser (~200 lines of code). Zero-copy, no dynamic allocation.	Massachusetts Institute of Technology (MIT)

Text processing

Library	Purpose	License
Onigmo	Regular expression library (fork of Oniguruma). Supports Perl 5.10+ expressions like \K, \R, and conditional patterns. Default regular expression engine for Ruby 2.0+.	BSD
tutf8e	Tiny UTF-8 encoder supporting ISO-8859-x and Windows-125x character set conversions.	Massachusetts Institute of Technology (MIT)
simdutf	SIMD-accelerated Unicode validation and transcoding (UTF-8, UTF-16, UTF-32).	Apache 2.0

Compression

Library	Purpose	License
miniz	Single-file deflate/inflate, zlib-subset implementation. Provides gzip compression.	Public Domain
Snappy	Fast compression/decompression library. Prioritizes speed over compression ratio.	BSD
zstd	Zstandard compression providing high compression ratios at fast speeds.	BSD

Scripting and webassembly

Library	Purpose	License
LuaJIT	Just-in-time (JIT) compiler for Lua. Powers the Lua filter plugin for custom record processing.	Massachusetts Institute of Technology (MIT)
WAMR	WebAssembly Micro Runtime from the Bytecode Alliance. Enables WASM filter plugins and WASM input plugins. Supports interpreter, AOT, and JIT execution modes.	Apache 2.0

Integration libraries

Library	Purpose	License
librdkafka	Apache Kafka C/C++ client library. Enables the Kafka input and Kafka output plugins. Supports producer, consumer, and admin clients with high performance (millions of messages/second).	BSD-2
Avro	Apache Avro data serialization system.	Apache 2.0

Storage and utilities

Library	Purpose	License
SQLite	Self-contained SQL database engine. Used for persistent storage operations.	Public Domain
jemalloc	General-purpose memory allocator emphasizing fragmentation avoidance and scalable concurrency. Optional alternative to system malloc.	BSD-2
lwrb	Lightweight ring buffer implementation. Thread-safe FIFO buffer with zero-copy DMA support.	Massachusetts Institute of Technology (MIT)
mpack	Simple and fast MessagePack encoder/decoder.	Massachusetts Institute of Technology (MIT)
libbacktrace	Library for producing symbolic back traces. Used for debugging and error reporting.	BSD

Version information

When Fluent Bit starts, it logs the versions of key libraries. For example, as of Fluent Bit 4.2:

[2025/01/15 10:30:00] [ info] [chunkio ] version=1.5.4
[2025/01/15 10:30:00] [ info] [ctraces ] version=1.0.5
[2025/01/15 10:30:00] [ info] [monkey  ] version=1.8.5
[2025/01/15 10:30:00] [ info] [wamr    ] version=2.4.1

Build options

Some libraries are optional and can be enabled or disabled at build time:

CMake option	Library	Default
FLB_WAMRC	WAMR AOT compiler	Off
FLB_LUAJIT	LuaJIT	On
FLB_OUT_KAFKA	librdkafka	Off
FLB_JEMALLOC	jemalloc	Off (platform dependent)

For more information on build options, see Build and install.