Key Concepts
Learn these key concepts to understand how Fluent Bit operates.
Last updated
Was this helpful?
Learn these key concepts to understand how Fluent Bit operates.
Last updated
Was this helpful?
Before diving into you might want to get acquainted with some of the key concepts of the service. This document provides an introduction to those concepts and common terminology. Reading this document will help you gain a more general understanding of the following topics:
Event or Record
Filtering
Tag
Timestamp
Match
Structured Message
Every incoming piece of data that belongs to a log or a metric that's retrieved by Fluent Bit is considered an Event or a Record.
As an example, consider the following content of a Syslog file:
It contains four lines that represent four independent Events.
An Event is comprised of:
timestamp
key/value metadata (v2.1.0 and greater)
payload
The Fluent Bit wire protocol represents an Event as a two-element array with a nested array as the first element:
where
TIMESTAMP is a timestamp in seconds as an integer or floating point value (not a string).
METADATA is an object containing event metadata, and might be empty.
MESSAGE is an object containing the event body.
Fluent Bit versions prior to v2.1.0 used:
to represent events. This format is still supported for reading input event streams.
Use filtering to:
Append specific information to the Event like an IP address or metadata.
Select a specific piece of the Event content.
Drop Events that match a certain pattern.
The timestamp represents the time an Event was created. Every Event contains an associated timestamps. All events have timestamps, and they're set by the input plugin or discovered through a data parsing process.
The timestamp is a numeric fractional integer in the format:
where:
_SECONDS_ is the number of seconds that have elapsed since the Unix epoch.
_NANOSECONDS_ is a fractional second or one thousand-millionth of a second.
Fluent Bit lets you route your collected and processed Events to one or multiple destinations. A Match represents a rule to select Events where a Tag matches a defined rule.
Source events can have a structure. A structure defines a set of keys and values inside the Event message to implement faster operations on data modifications. Fluent Bit treats every Event message as a structured message.
Consider the following two messages:
No structured message
With a structured message
You might need to perform modifications on an Event's content. The process to alter, append to, or drop Events is called .
Every Event ingested by Fluent Bit is assigned a Tag. This tag is an internal string used in a later stage by the Router to decide which Filter or phase it must go through.
Most tags are assigned manually in the configuration. If a tag isn't specified, Fluent Bit assigns the name of the plugin instance where that Event was generated from.
The input plugin doesn't assign tags. This plugin speaks the Fluentd wire protocol called Forward where every Event already comes with a Tag associated. Fluent Bit will always use the incoming Tag set by the client.
A tagged record must always have a Matching rule. To learn more about Tags and Matches, see .
To learn more about Tags and Matches, see .
For performance reasons, Fluent Bit uses a binary serialization data format called .