Concatenate Multiline or Stack trace log messages. Available on Fluent Bit >= v1.8.2.
The Multiline Filter helps to concatenate messages that originally belong to one context but were split across multiple records or log lines. Common examples are stack traces or applications that print logs in multiple lines.
As part of the built-in functionality, without major configuration effort, you can enable one of ours built-in parsers with auto detection and multi format support:
The usage of this filter depends on a previous configuration of a Multiline Parser definition.
If you aim to concatenate messages split originally by Docker or CRI container engines, we recommend doing the concatenation on Tail plugin, this same functionality exists there.
This filter does not perform buffering that persists across different Chunks. This filter process one Chunk at a time and is not suitable for sources that might send multiline messages in separated chunks.
For cases where Multiline mode is required and the source plugin does not support it, please file a Github Enhancement with such requirement and specific details of the use case.
Configuration Parameters
The plugin supports the following configuration parameters:
Property
Description
multiline.parser
Specify one or multiple Multiline Parser definitions to apply to the content. You can specify multiple multiline parsers to detect different formats by separating them with a comma.
multiline.key_content
Key name that holds the content to process. Note that a Multiline Parser definition can already specify the key_content to use, but this option allows to overwrite that value for the purpose of the filter.
Configuration Example
The following example aims to parse a log file called test.log that contains some full lines, a custom Java stacktrace and a Go stacktrace.
This is the primary Fluent Bit configuration file. It includes the parsers_multiline.conf and tails the file test.log by applying the multiline parsers multiline-regex-test and go. Then it sends the processing to the standard output.
[SERVICE]
flush 1
log_level info
parsers_file parsers_multiline.conf
[INPUT]
name tail
path test.log
read_from_head true
[FILTER]
name multiline
match *
multiline.key_content log
multiline.parser go, multiline-regex-test
[OUTPUT]
name stdout
match *
This second file defines a multiline parser for the example. Note that a second multiline parser called go is used in fluent-bit.conf, but this one is a built-in parser.
[MULTILINE_PARSER]
name multiline-regex-test
type regex
flush_timeout 1000
#
# Regex rules for multiline parsing
# ---------------------------------
#
# configuration hints:
#
# - first state always has the name: start_state
# - every field in the rule must be inside double quotes
#
# rules | state name | regex pattern | next state
# ------|---------------|--------------------------------------------
rule "start_state" "/(Dec \d+ \d+\:\d+\:\d+)(.*)/" "cont"
rule "cont" "/^\s+at.*/" "cont"
An example file with multiline and multiformat content:
The lines that did not match a pattern are not considered as part of the multiline message, while the ones that matched the rules were concatenated properly.
