Grep

The Grep Filter plugin lets you match or exclude specific records based on regular expression patterns for values or nested values.

Configuration parameters

The plugin supports the following configuration parameters:

Record Accessor Enabled

Enable the feature to specify the KEY. Use the record accessor to match values against nested values.

Filter records

To start filtering records, run the filter from the command line or through the configuration file. The following example assumes that you have a file named lines.txt with the following content:

Command line

When using the command line, pay close attention to quote the regular expressions. Using a configuration file might be easier.

The following command loads the plugin and reads the content of lines.txt. Then the grep filter applies a regular expression rule over the log field created by the tail plugin and only passes records with a field value starting with aa:

Configuration file

The filter lets you use multiple rules which are applied in order. You can have as many Regex and Exclude entries as required.

Nested fields example

To match or exclude records based on nested values, you can use format as the KEY name.

Consider the following record example:

For example, to exclude records that match the nested field kubernetes.labels.app, use the following rule:

Excluding records with missing or invalid fields

You might want to drop records that are missing certain keys.

One way to do this is to exclude with a regex that matches anything. A missing key fails this check.

The followinfg example checks for a specific valid value for the key:

The specified key iot_timestamp must match the expected expression. If it doesn't, or is missing or empty, then it will be excluded.

Multiple conditions

If you want to set multiple Regex or Exclude, use the Logical_Op property to use a logical conjuction or disjunction.

If Logical_Op is set, setting both Regex and Exclude results in an error.

The output looks similar to: