1.9
Installation
Administration
Data Pipeline
Stream Processing
Record Modifier
The Record Modifier Filter plugin allows to append fields or to exclude specific fields.
Configuration Parameters
The plugin supports the following configuration parameters: Remove_key and Allowlist_key are exclusive.
Key
Description
Record
Append fields. This parameter needs key and value pair.
Remove_key
If the key is matched, that field is removed.
Allowlist_key
If the key is not matched, that field is removed.
Whitelist_key
An alias of
Allowlist_key for backwards compatibility.Getting Started
In order to start filtering records, you can run the filter from the command line or through the configuration file.
This is a sample in_mem record to filter.
1
{"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>299352, "Swap.total"=>2064380, "Swap.used"=>32656, "Swap.free"=>2031724}
Copied!
Append fields
The following configuration file is to append product name and hostname (via environment variable) to record.
1
[INPUT]
2
Name mem
3
Tag mem.local
4
​
5
[OUTPUT]
6
Name stdout
7
Match *
8
​
9
[FILTER]
10
Name record_modifier
11
Match *
12
Record hostname ${HOSTNAME}
13
Record product Awesome_Tool
Copied!
You can also run the filter from command line.
1
$ fluent-bit -i mem -o stdout -F record_modifier -p 'Record=hostname ${HOSTNAME}' -p 'Record=product Awesome_Tool' -m '*'
Copied!
The output will be
1
[0] mem.local: [1492436882.000000000, {"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>299352, "Swap.total"=>2064380, "Swap.used"=>32656, "Swap.free"=>2031724, "hostname"=>"localhost.localdomain", "product"=>"Awesome_Tool"}]
Copied!
Remove fields with Remove_key
The following configuration file is to remove 'Swap.*' fields.
1
[INPUT]
2
Name mem
3
Tag mem.local
4
​
5
[OUTPUT]
6
Name stdout
7
Match *
8
​
9
[FILTER]
10
Name record_modifier
11
Match *
12
Remove_key Swap.total
13
Remove_key Swap.used
14
Remove_key Swap.free
Copied!
You can also run the filter from command line.
1
$ fluent-bit -i mem -o stdout -F record_modifier -p 'Remove_key=Swap.total' -p 'Remove_key=Swap.free' -p 'Remove_key=Swap.used' -m '*'
Copied!
The output will be
1
[0] mem.local: [1492436998.000000000, {"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>295332}]
Copied!
Remove fields with Allowlist_key
The following configuration file is to remain 'Mem.*' fields.
1
[INPUT]
2
Name mem
3
Tag mem.local
4
​
5
[OUTPUT]
6
Name stdout
7
Match *
8
​
9
[FILTER]
10
Name record_modifier
11
Match *
12
Allowlist_key Mem.total
13
Allowlist_key Mem.used
14
Allowlist_key Mem.free
Copied!
You can also run the filter from command line.
1
$ fluent-bit -i mem -o stdout -F record_modifier -p 'Allowlist_key=Mem.total' -p 'Allowlist_key=Mem.free' -p 'Allowlist_key=Mem.used' -m '*'
Copied!
The output will be
1
[0] mem.local: [1492436998.000000000, {"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>295332}]
Copied!
Last modified 4mo ago
Export as PDF
Copy link