InfluxDB

The influxdb output plugin, allows to flush your records into a InfluxDB time series database. The following instructions assumes that you have a fully operational InfluxDB service running in your system.
Configuration Parameters
Key
Description
default
Host
IP address or hostname of the target InfluxDB service
127.0.0.1
Port
TCP port of the target InfluxDB service
8086
Database
InfluxDB database name where records will be inserted
fluentbit
Bucket
InfluxDB bucket name where records will be inserted - if specified, database is ignored and v2 of API is used
​
Org
InfluxDB organization name where the bucket is (v2 only)
fluent
Sequence_Tag
The name of the tag whose value is incremented for the consecutive simultaneous events.
_seq
HTTP_User
Optional username for HTTP Basic Authentication
​
HTTP_Passwd
Password for user defined in HTTP_User
​
HTTP_Token
Authentication token used with InfluDB v2 - if specified, both HTTP_User and HTTP_Passwd are ignored
​
Tag_Keys
Space separated list of keys that needs to be tagged
​
Auto_Tags
Automatically tag keys where value is string. This option takes a boolean value: True/False, On/Off.
Off
Tags_List_Enabled
Dynamically tag keys which are in the string array at Tags_List_Key key. This option takes a boolean value: True/False, On/Off.
Off
Tags_List_Key
Key of the string array optionally contained within each log record that contains tag keys for that record
tags
TLS / SSL
InfluxDB output plugin supports TTL/SSL, for more details about the properties available and general configuration, please refer to the TLS/SSL section.
Getting Started
In order to start inserting records into an InfluxDB service, you can run the plugin from the command line or through the configuration file:
Command Line
The influxdb plugin, can read the parameters from the command line in two ways, through the -p argument (property) or setting them directly through the service URI. The URI format is the following:
1
influxdb://host:port
Copied!
Using the format specified, you could start Fluent Bit through:
1
$ fluent-bit -i cpu -t cpu -o influxdb://127.0.0.1:8086 -m '*'
Copied!
Configuration File
In your main configuration file append the following Input & Output sections:
1
[INPUT]
2
    Name  cpu
3
    Tag   cpu
4
​
5
[OUTPUT]
6
    Name          influxdb
7
    Match         *
8
    Host          127.0.0.1
9
    Port          8086
10
    Database      fluentbit
11
    Sequence_Tag  _seq
Copied!
Tagging
Basic example of Tag_Keys usage:
1
[INPUT]
2
    Name            tail
3
    Tag             apache.access
4
    parser          apache2
5
    path            /var/log/apache2/access.log
6
​
7
[OUTPUT]
8
    Name          influxdb
9
    Match         *
10
    Host          127.0.0.1
11
    Port          8086
12
    Database      fluentbit
13
    Sequence_Tag  _seq
14
    # make tags from method and path fields
15
    Tag_Keys      method path
Copied!
With Auto_Tags=On in this example cause error, because every parsed field value type is string. Best usage of this option in metrics like record where one or more field value is not string typed.
Basic example of Tags_List_Key usage:
1
[INPUT]
2
    Name              dummy
3
    # tagged fields: level, ID, businessObjectID, status
4
    Dummy             {"msg": "Transfer completed", "level": "info", "ID": "1234", "businessObjectID": "qwerty", "status": "OK", "tags": ["ID", "businessObjectID"]}
5
​
6
[OUTPUT]
7
    Name          influxdb
8
    Match         *
9
    Host          127.0.0.1
10
    Port          8086
11
    Bucket        My_Bucket
12
    Org           My_Org
13
    Sequence_Tag  _seq
14
    HTTP_Token    My_Token
15
    # tag all fields inside tags string array
16
    Tags_List_Enabled True
17
    Tags_List_Key tags
18
    # tag level, status fields
19
    Tag_Keys level status
Copied!
Testing
Before to start Fluent Bit, make sure the target database exists on InfluxDB, using the above example, we will insert the data into a fluentbit database.
1. Create database
Log into InfluxDB console:
1
$ influx
2
Visit https://enterprise.influxdata.com to register for updates, InfluxDB server management, and monitoring.
3
Connected to http://localhost:8086 version 1.1.0
4
InfluxDB shell version: 1.1.0
5
>
Copied!
Create the database:
1
> create database fluentbit
2
>
Copied!
Check the database exists:
1
> show databases
2
name: databases
3
name
4
----
5
_internal
6
fluentbit
7
​
8
>
Copied!
2. Run Fluent Bit
The following command will gather CPU metrics from the system and send the data to InfluxDB database every five seconds:
1
$ bin/fluent-bit -i cpu -t cpu -o influxdb -m '*'
Copied!
Note that all records coming from the cpu input plugin, have a tag cpu, this tag is used to generate the measurement in InfluxDB
3. Query the data
From InfluxDB console, choose your database:
1
> use fluentbit
2
Using database fluentbit
Copied!
Now query some specific fields:
1
> SELECT cpu_p, system_p, user_p FROM cpu
2
name: cpu
3
time                  cpu_p   system_p    user_p
4
----                  -----   --------    ------
5
1481132860000000000   2.75        0.5      2.25
6
1481132861000000000   2           0.5      1.5
7
1481132862000000000   4.75        1.5      3.25
8
1481132863000000000   6.75        1.25     5.5
9
1481132864000000000   11.25       3.75     7.5
Copied!
The CPU input plugin gather more metrics per CPU core, in the above example we just selected three specific metrics. The following query will give a full result:
1
> SELECT * FROM cpu
Copied!
4. View tags
Query tagged keys:
1
> SHOW TAG KEYS ON fluentbit FROM "apache.access"
2
name: apache.access
3
tagKey
4
------
5
_seq
6
method
7
path
Copied!
And now query method key values:
1
> SHOW TAG VALUES ON fluentbit FROM "apache.access" WITH KEY = "method"
2
name: apache.access
3
key    value
4
---    -----
5
method "MATCH"
6
method "POST"
Copied!

HTTP

Kafka

Last modified 15d ago

Export as PDF

Copy link

Contents

Configuration Parameters