Parsers

You can define customer parsers in the parsers section of YAML configuration files.

To define custom multiline parsers, use the multiline_parsers section of YAML configuration files.

Syntax

To define custom parsers in the parsers section of a YAML configuration file, use the following syntax.

fluent-bit.yaml

parsers:
  - name: custom_parser1
    format: json
    time_key: time
    time_format: '%Y-%m-%dT%H:%M:%S.%L'
    time_keep: on

  - name: custom_parser2
    format: regex
    regex: '^\<(?<pri>[0-9]{1,5})\>1 (?<time>[^ ]+) (?<host>[^ ]+) (?<ident>[^ ]+) (?<pid>[-0-9]+) (?<msgid>[^ ]+) (?<extradata>(\[(.*)\]|-)) (?<message>.+)$'
    time_key: time
    time_format: '%Y-%m-%dT%H:%M:%S.%L'
    time_keep: on
    types: pid:integer

For information about supported configuration options for custom parsers, see configuring custom parsers.

Standalone parsers files

In addition to defining parsers in the parsers section of YAML configuration files, you can store parser definitions in standalone files. These standalone files require the same syntax as parsers defined in a standard YAML configuration file.

To add a standalone parsers file to Fluent Bit, use the parsers_file parameter in the service section of your YAML configuration file.

Add a standalone parsers file to Fluent Bit

To add a standalone parsers file to Fluent Bit, follow these steps.

1. Define custom parsers in a standalone YAML file. For example, custom-parsers.yaml defines two custom parsers:

custom-parsers.yaml

parsers:
  - name: custom_parser1
    format: json
    time_key: time
    time_format: '%Y-%m-%dT%H:%M:%S.%L'
    time_keep: on

  - name: custom_parser2
    format: regex
    regex: '^\<(?<pri>[0-9]{1,5})\>1 (?<time>[^ ]+) (?<host>[^ ]+) (?<ident>[^ ]+) (?<pid>[-0-9]+) (?<msgid>[^ ]+) (?<extradata>(\[(.*)\]|-)) (?<message>.+)$'
    time_key: time
    time_format: '%Y-%m-%dT%H:%M:%S.%L'
    time_keep: on
    types: pid:integer

1. Update the parsers_file parameter in the service section of your YAML configuration file:

fluent-bit.yaml

service:
  parsers_file: my-parsers.yaml