Fluent Bit: Official Manual
Slack
GitHub
Discussions
Community Meetings
Search…
1.9
Fluent Bit v1.9 Documentation
About
What is Fluent Bit?
A Brief History of Fluent Bit
Fluentd & Fluent Bit
License
Concepts
Key Concepts
Buffering
Data Pipeline
Installation
Getting Started with Fluent Bit
Upgrade Notes
Supported Platforms
Requirements
Sources
Linux Packages
Docker
Containers on AWS
Amazon EC2
Kubernetes
macOS
Windows
Yocto / Embedded Linux
Administration
Configuring Fluent Bit
Security
Buffering & Storage
Backpressure
Scheduling and Retries
Networking
Memory Management
Monitoring
Dump Internals / Signal
HTTP Proxy
Local Testing
Validating your Data and Structure
Running a Logging Pipeline Locally
Data Pipeline
Pipeline Monitoring
Inputs
Parsers
Filters
AWS Metadata
CheckList
Expect
GeoIP2 Filter
Grep
Kubernetes
Lua
Parser
Record Modifier
Modify
Multiline
Nest
Nightfall
Rewrite Tag
Standard Output
Throttle
Tensorflow
Outputs
Stream Processing
Introduction to Stream Processing
Overview
Changelog
Getting Started
Fluent Bit for Developers
C Library API
Ingest Records Manually
Golang Output Plugins
Developer guide for beginners on contributing to Fluent Bit
Powered By
GitBook
Record Modifier
The
Record Modifier Filter
plugin allows to append fields or to exclude specific fields.
Configuration Parameters
The plugin supports the following configuration parameters:
Remove_key
and
Allowlist_key
are exclusive.
Key
Description
Record
Append fields. This parameter needs key and value pair.
Remove_key
If the key is matched, that field is removed.
Allowlist_key
If the key is
not
matched, that field is removed.
Whitelist_key
An alias of
Allowlist_key
for backwards compatibility.
Getting Started
In order to start filtering records, you can run the filter from the command line or through the configuration file.
This is a sample in_mem record to filter.
1
{"Mem.total"=>1016024, "Mem.used"=>716672, "Mem.free"=>299352, "Swap.total"=>2064380, "Swap.used"=>32656, "Swap.free"=>2031724}
Copied!
Append fields
The following configuration file is to append product name and hostname (via environment variable) to record.
1
[
INPUT
]
2
Name mem
3
Tag mem
.
local
4
​
5
[
OUTPUT
]
6
Name stdout
7
Match
*
8
​
9
[
FILTER
]
10
Name record_modifier
11
Match
*
12
Record hostname $
{
HOSTNAME
}
13
Record product Awesome_Tool
Copied!
You can also run the filter from command line.
1
$ fluent-bit -i mem -o stdout -F record_modifier -p 'Record=hostname ${HOSTNAME}' -p 'Record=product Awesome_Tool' -m '*'
Copied!
The output will be
1
[
0
]
mem
.
local
:
[
1492436882.000000000
,
{
"Mem.total"
=>
1016024
,
"Mem.used"
=>
716672
,
"Mem.free"
=>
299352
,
"Swap.total"
=>
2064380
,
"Swap.used"
=>
32656
,
"Swap.free"
=>
2031724
,
"hostname"
=>
"localhost.localdomain"
,
"product"
=>
"Awesome_Tool"
}]
Copied!
Remove fields with Remove_key
The following configuration file is to remove 'Swap.*' fields.
1
[
INPUT
]
2
Name mem
3
Tag mem
.
local
4
​
5
[
OUTPUT
]
6
Name stdout
7
Match
*
8
​
9
[
FILTER
]
10
Name record_modifier
11
Match
*
12
Remove_key Swap
.
total
13
Remove_key Swap
.
used
14
Remove_key Swap
.
free
Copied!
You can also run the filter from command line.
1
$ fluent-bit -i mem -o stdout -F record_modifier -p 'Remove_key=Swap.total' -p 'Remove_key=Swap.free' -p 'Remove_key=Swap.used' -m '*'
Copied!
The output will be
1
[
0
]
mem
.
local
:
[
1492436998.000000000
,
{
"Mem.total"
=>
1016024
,
"Mem.used"
=>
716672
,
"Mem.free"
=>
295332
}]
Copied!
Remove fields with Allowlist_key
The following configuration file is to remain 'Mem.*' fields.
1
[
INPUT
]
2
Name mem
3
Tag mem
.
local
4
​
5
[
OUTPUT
]
6
Name stdout
7
Match
*
8
​
9
[
FILTER
]
10
Name record_modifier
11
Match
*
12
Allowlist_key Mem
.
total
13
Allowlist_key Mem
.
used
14
Allowlist_key Mem
.
free
Copied!
You can also run the filter from command line.
1
$ fluent-bit -i mem -o stdout -F record_modifier -p 'Allowlist_key=Mem.total' -p 'Allowlist_key=Mem.free' -p 'Allowlist_key=Mem.used' -m '*'
Copied!
The output will be
1
[
0
]
mem
.
local
:
[
1492436998.000000000
,
{
"Mem.total"
=>
1016024
,
"Mem.used"
=>
716672
,
"Mem.free"
=>
295332
}]
Copied!
Previous
Parser
Next
Modify
Last modified
4mo ago
Export as PDF
Copy link
Contents
Configuration Parameters
Getting Started
Append fields
Remove fields with Remove_key
Remove fields with Allowlist_key