Splunk

Supported event types: logs

The Splunk input plugin handles Splunk HTTP HEC requests.

Configuration parameters

This plugin uses the following configuration parameters:

The table below includes both:

• settings specific to the Splunk input plugin

• shared http_server.* listener settings that are used by several HTTP-based inputs

For a cross-plugin explanation of the shared listener settings, see Shared HTTP listener settings for inputs.

Key	Description	Default
add_remote_addr	Inject a remote address field into the record, using the X-Forwarded-For header or the connection address as the value.	false
buffer_chunk_size	Set the chunk size for incoming JSON messages. These chunks are then stored and managed in the space available by buffer_max_size. Compatibility alias for http_server.buffer_chunk_size.	512K
buffer_max_size	Set the maximum buffer size to receive a JSON message. Compatibility alias for http_server.buffer_max_size.	4M
http2	Enable HTTP/2 support. Compatibility alias for http_server.http2.	true
http_server.max_connections	Maximum number of concurrent active HTTP connections. 0 means unlimited.	0
http_server.workers	Number of HTTP listener worker threads.	1
http_server.ingress_queue_event_limit	Maximum number of deferred ingress queue entries. Applies only when http_server.workers is greater than 1.	8192
http_server.ingress_queue_byte_limit	Maximum size of the deferred ingress queue. Applies only when http_server.workers is greater than 1.	256M
listen	The address to listen on.	0.0.0.0
port	The port for Fluent Bit to listen on.	8088
remote_addr_key	Record key name used to store the remote address when add_remote_addr is enabled.	remote_addr
splunk_token	Specify a Splunk token for HTTP HEC authentication. If multiple tokens are specified (with commas and no spaces), usage will be divided across each of the tokens.	none
splunk_token_key	Set a record key for storing the Splunk token for HTTP HEC. Use only when store_token_in_metadata is false.	@splunk_token
store_token_in_metadata	Store Splunk HEC tokens in the Fluent Bit metadata. If set to false, they will be stored as key-value pairs in the record data.	true
success_header	Add an HTTP header key/value pair on success. Multiple headers can be set.	none
tag_key	Specify the key name to overwrite a tag. If set, the tag will be overwritten by a value of the key.	none
threaded	Indicates whether to run this input in its own thread.	false

The http_server.ingress_queue_event_limit and http_server.ingress_queue_byte_limit settings matter only when http_server.workers is greater than 1.

Get started

To get started, you can run the plugin from the command line or through the configuration file.

Set a tag

The tag for the Splunk input plugin is set by adding the tag to the end of the request URL by default. This tag is then used to route the event through the system. The default behavior of the Splunk input sets the tags for the following endpoints:

• /services/collector

• /services/collector/event

• /services/collector/raw

The requests for these endpoints are interpreted as services_collector, services_collector_event, and services_collector_raw.

To use the other tags for multiple instantiating input Splunk plugins, you must specify the tag property on each Splunk plugin configuration to prevent data pipeline collisions.

Command line

From the command line you can configure Fluent Bit to handle HTTP HEC requests with the following options:

fluent-bit -i splunk -p port=8088 -o stdout

Configuration file

In your main configuration file append the following sections:

fluent-bit.yaml

pipeline:
  inputs:
    - name: splunk
      listen: 0.0.0.0
      port: 8088

  outputs:
    - name: stdout
      match: '*'

fluent-bit.conf

Authentication with HEC tokens

To require authentication, specify one or more Splunk HEC tokens. Multiple tokens can be provided as a comma-separated list:

fluent-bit.yaml

pipeline:
  inputs:
    - name: splunk
      port: 8088
      splunk_token: "my-secret-token,another-token"

  outputs:
    - name: stdout
      match: '*'

fluent-bit.conf

Add a remote address field

When add_remote_addr is set to true, a remote address field is injected into every record. The value is extracted from the X-Forwarded-For header, or falls back to the connection address if the header isn't present. Use remote_addr_key to customize the field name.

fluent-bit.yaml

pipeline:
  inputs:
    - name: splunk
      port: 8088
      add_remote_addr: true
      remote_addr_key: remote_addr

  outputs:
    - name: stdout
      match: '*'

fluent-bit.conf

Custom success headers

Use success_header to add custom HTTP headers to successful responses. Use this for CORS or other HTTP requirements:

fluent-bit.yaml

pipeline:
  inputs:
    - name: splunk
      port: 8088
      success_header: "X-Custom-Header myvalue"

  outputs:
    - name: stdout
      match: '*'

fluent-bit.conf