Multiline
Concatenate Multiline or Stack trace log messages. Available on Fluent Bit >= v1.8.2.
Last updated
Was this helpful?
Concatenate Multiline or Stack trace log messages. Available on Fluent Bit >= v1.8.2.
Last updated
Was this helpful?
The Multiline Filter helps to concatenate messages that originally belong to one context but were split across multiple records or log lines. Common examples are stack traces or applications that print logs in multiple lines.
As part of the built-in functionality, without major configuration effort, you can enable one of ours built-in parsers with auto detection and multi format support:
go
python
ruby
java (Google Cloud Platform Java stacktrace format)
Some comments about this filter:
The usage of this filter depends on a previous configuration of a definition.
If you aim to concatenate messages split originally by Docker or CRI container engines, we recommend doing the concatenation on , this same functionality exists there.
This filter does not perform buffering that persists across different Chunks. This filter process one Chunk at a time and is not suitable for sources that might send multiline messages in separated chunks.
For cases where Multiline mode is required and the source plugin does not support it, please file a Github Enhancement with such requirement and specific details of the use case.
The plugin supports the following configuration parameters:
multiline.parser
multiline.key_content
Key name that holds the content to process. Note that a Multiline Parser definition can already specify the key_content
to use, but this option allows to overwrite that value for the purpose of the filter.
The following example aims to parse a log file called test.log
that contains some full lines, a custom Java stacktrace and a Go stacktrace.
Example files content:
This is the primary Fluent Bit configuration file. It includes the parsers_multiline.conf
and tails the file test.log
by applying the multiline parsers multiline-regex-test
and go
. Then it sends the processing to the standard output.
By running Fluent Bit with the given configuration file you will obtain:
The lines that did not match a pattern are not considered as part of the multiline message, while the ones that matched the rules were concatenated properly.
Specify one or multiple to apply to the content. You can specify multiple multiline parsers to detect different formats by separating them with a comma.
The following example files can be located at: