For the complete documentation index, see llms.txt. This page is also available as Markdown.

Output Formats

Some output plugins have a format configuration parameter. Supported values vary by plugin:

Plugin
Supported formats

HTTP

gelf, json, json_lines, json_stream, msgpack

Kafka

avro (build-dependent), gelf, json, msgpack, raw

Standard Output

json, json_lines, json_stream, msgpack

TCP/TLS

json, json_lines, json_stream, msgpack

UDP

json, json_lines, json_stream, msgpack

WebSocket

gelf, json, json_lines, json_stream, msgpack

Some plugins, such as Kafka, support additional formats not listed here (for example, avro and raw). Always check the individual plugin documentation for the full list of supported values.

The format specifications and references to external documentation are available in this page.

JSON formats

The most custom format options are the JSON types. The following are common variants of sending JSON encoded data over various APIs.

The following example payloads are modified for readability.

Records shown in example payloads are the three first commits of the Fluent Bit repository.

Nesting depth limit

In Fluent Bit version 5.1.3 and greater, converting a record to any of these JSON formats serializes at most 512 levels of nested maps and arrays. Maps and arrays through level 512 are preserved. When a record nests deeper than this limit, Fluent Bit writes each non-empty map or array at level 513 as null, omits its contents, and keeps the rest of the record intact. An empty map or array at level 513 is still written as {} or [].

Fluent Bit logs the following warning at most once for each serialization pass, no matter how many branches are affected:

[warn] [pack] msgpack to JSON conversion exceeded the maximum nesting depth (512), truncating remaining structure

With json, a pass covers the full array of queued records. With json_lines and json_stream, a pass covers one record. If the output buffer is too small, Fluent Bit grows it and retries in a new pass, which can log the warning again.

json

Queued records encoded as a single JSON array of JSON encoded Fluent Bit records.

Example payload

json_stream

a.k.a. Concatenated JSON

Queued JSON-encoded Fluent Bit records, with no separator between records.

Example json_stream payload

json_lines

Also known as newline-delimited JSON.

Queued JSON-encoded Fluent Bit records, separated by line-breaks (\n).

Example json_lines payload

msgpack

MessagePack is a binary format for serializing objects.

Refer to the official MessagePack format specification.

GELF

Graylog Extended Log Format (GELF) is a JSON based log format which is designed to avoid the shortcomings of classic plain syslog.

See the official documentation for the GELF format.

It uses a fixed set of keys, with the addition of adding custom keys prefixed by one single underscore character.

Example GELF payload

Here is an example payload, copied directly from the official documentation.

Last updated

Was this helpful?