Record Modifier

The Record Modifier Filter plugin allows to append fields or to exclude specific fields.
Configuration Parameters
The plugin supports the following configuration parameters: Remove_key and Allowlist_key are exclusive.
Key
Description
Record
Append fields. This parameter needs key and value pair.
Remove_key
If the key is matched, that field is removed.
Allowlist_key
If the key is not matched, that field is removed.
Whitelist_key
An alias of Allowlist_key for backwards compatibility.
Getting Started
In order to start filtering records, you can run the filter from the command line or through the configuration file.
This is a sample in_mem record to filter.
1
{"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>299352, "Swap.total"=>2064380, "Swap.used"=>32656, "Swap.free"=>2031724}
Copied!
Append fields
The following configuration file is to append product name and hostname (via environment variable) to record.
1
[INPUT]
2
    Name mem
3
    Tag  mem.local
4
​
5
[OUTPUT]
6
    Name  stdout
7
    Match *
8
​
9
[FILTER]
10
    Name record_modifier
11
    Match *
12
    Record hostname ${HOSTNAME}
13
    Record product Awesome_Tool
Copied!
You can also run the filter from command line.
1
$ fluent-bit -i mem -o stdout -F record_modifier -p 'Record=hostname ${HOSTNAME}' -p 'Record=product Awesome_Tool' -m '*'
Copied!
The output will be
1
[0] mem.local: [1492436882.000000000, {"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>299352, "Swap.total"=>2064380, "Swap.used"=>32656, "Swap.free"=>2031724, "hostname"=>"localhost.localdomain", "product"=>"Awesome_Tool"}]
Copied!
Remove fields with Remove_key
The following configuration file is to remove 'Swap.*' fields.
1
[INPUT]
2
    Name mem
3
    Tag  mem.local
4
​
5
[OUTPUT]
6
    Name  stdout
7
    Match *
8
​
9
[FILTER]
10
    Name record_modifier
11
    Match *
12
    Remove_key Swap.total
13
    Remove_key Swap.used
14
    Remove_key Swap.free
Copied!
You can also run the filter from command line.
1
$ fluent-bit -i mem -o stdout -F  record_modifier -p 'Remove_key=Swap.total' -p 'Remove_key=Swap.free' -p 'Remove_key=Swap.used' -m '*'
Copied!
The output will be
1
[0] mem.local: [1492436998.000000000, {"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>295332}]
Copied!
Remove fields with Allowlist_key
The following configuration file is to remain 'Mem.*' fields.
1
[INPUT]
2
    Name mem
3
    Tag  mem.local
4
​
5
[OUTPUT]
6
    Name  stdout
7
    Match *
8
​
9
[FILTER]
10
    Name record_modifier
11
    Match *
12
    Allowlist_key Mem.total
13
    Allowlist_key Mem.used
14
    Allowlist_key Mem.free
Copied!
You can also run the filter from command line.
1
$ fluent-bit -i mem -o stdout -F  record_modifier -p 'Allowlist_key=Mem.total' -p 'Allowlist_key=Mem.free' -p 'Allowlist_key=Mem.used' -m '*'
Copied!
The output will be
1
[0] mem.local: [1492436998.000000000, {"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>295332}]
Copied!

Parser

Modify

Last modified 6mo ago

Export as PDF

Copy link

Contents

Configuration Parameters

Getting Started

Append fields

Remove fields with Remove_key

Remove fields with Allowlist_key