The kafka-rest output plugin, allows to flush your records into a Kafka REST Proxy server. The following instructions assumes that you have a fully operational Kafka REST Proxy and Kafka services running in your environment.
Kafka REST Proxy output plugin supports TTL/SSL, for more details about the properties available and general configuration, please refer to the section.
In order to insert records into a Kafka REST Proxy service, you can run the plugin from the command line or through the configuration file:
The kafka-rest plugin, can read the parameters from the command line in two ways, through the -p argument (property), e.g:
In your main configuration file append the following Input & Output sections:
Host
IP address or hostname of the target Kafka REST Proxy server
127.0.0.1
Port
TCP port of the target Kafka REST Proxy server
8082
Topic
Set the Kafka topic
fluent-bit
Partition
Set the partition number (optional)
Message_Key
Set a message key (optional)
Time_Key
The Time_Key property defines the name of the field that holds the record timestamp.
@timestamp
Time_Key_Format
Defines the format of the timestamp.
%Y-%m-%dT%H:%M:%S
Include_Tag_Key
Append the Tag name to the final record.
Off
Tag_Key
If Include_Tag_Key is enabled, this property defines the key name for the tag.
_flb-key
$ fluent-bit -i cpu -t cpu -o kafka-rest -p host=127.0.0.1 -p port=8082 -m '*'[INPUT]
Name cpu
Tag cpu
[OUTPUT]
Name kafka-rest
Match *
Host 127.0.0.1
Port 8082
Topic fluent-bit
Message_Key my_key