CheckList

The following plugin looks up if a value in a specified list exists and then allows the addition of a record to indicate if found. Introduced in version 1.8.4

Configuration Parameters

The plugin supports the following configuration parameters

Key

Description

file

The single value file that Fluent Bit will use as a lookup table to determine if the specified lookup_key exists

lookup_key

The specific key to look up and determine if it exists, supports record accessor

record

The record to add if the lookup_key is found in the specified file. Note you may add multiple record parameters.

Example Configuration

[INPUT]
name tail
tag test1
path test1.log
read_from_head true
parser json
[FILTER]
name checklist
match test1
file ip_list.txt
lookup_key $remote_addr
record ioc abc
record badurl null
log_level debug
[OUTPUT]
name stdout
match test1

In the following configuration we will read a file test1.log that includes the following values

{"remote_addr": true, "ioc":"false", "url":"https://badurl.com/payload.htm","badurl":"no"}
{"remote_addr": "7.7.7.2", "ioc":"false", "url":"https://badurl.com/payload.htm","badurl":"no"}
{"remote_addr": "7.7.7.3", "ioc":"false", "url":"https://badurl.com/payload.htm","badurl":"no"}
{"remote_addr": "7.7.7.4", "ioc":"false", "url":"https://badurl.com/payload.htm","badurl":"no"}
{"remote_addr": "7.7.7.5", "ioc":"false", "url":"https://badurl.com/payload.htm","badurl":"no"}
{"remote_addr": "7.7.7.6", "ioc":"false", "url":"https://badurl.com/payload.htm","badurl":"no"}
{"remote_addr": "7.7.7.7", "ioc":"false", "url":"https://badurl.com/payload.htm","badurl":"no"}

Additionally, we will use the following lookup file which contains a list of malicious IPs (ip_list.txt)

1.2.3.4
6.6.4.232
7.7.7.7

In the configuration we are using $remote_addr as the lookup key and 7.7.7.7 is malicious. This means the record we would output for the last record would look like the following

{"remote_addr": "7.7.7.7", "ioc":"abc", "url":"https://badurl.com/payload.htm","badurl":"null"}