HTTP
The HTTP output plugin lets you flush your records into an HTTP endpoint. It issues POST, PUT, or PATCH requests with the data records in MessagePack (or JSON) format.
Configuration parameters
allow_duplicated_headers
Specify if duplicated headers are allowed. If a duplicated header is found, the latest key/value set is preserved.
true
aws_auth
Enable AWS SigV4 authentication.
false
aws_external_id
External ID for the AWS IAM Role specified with aws_role_arn, used by SigV4 authentication.
none
aws_profile
AWS Profile name. AWS Profiles can be configured with AWS CLI and are usually stored in $HOME/.aws/ directory.
none
aws_region
Specify the AWS region of your service, used by SigV4 authentication.
none
aws_role_arn
AWS IAM Role to assume, used by SigV4 authentication.
none
aws_service
Specify the AWS service code of your service, used by SigV4 authentication (for example, es, xray). Usually found in the service endpoint's subdomains, protocol://service-code.region-code.amazonaws.com.
none
aws_sts_endpoint
Specify the custom STS endpoint to be used by STS API. Used with the aws_role_arn option and by SigV4 authentication.
none
body_key
Specify the key to use as the body of the request (must prefix with $). The key must contain either a binary or raw string, and the content type can be specified using headers_key, which must be passed whenever body_key is present. When this option is present, each msgpack record will create a separate request.
none
compress
Set payload compression mechanism. Allowed values: gzip, snappy, zstd.
none
format
Specify the data format to be used in the HTTP request body. Supported formats: gelf, json, json_stream, json_lines, msgpack.
json
gelf_full_message_key
Specify the key to use for the full message in gelf format.
none
gelf_host_key
Specify the key to use for the host in gelf format.
none
gelf_level_key
Specify the key to use for the level in gelf format.
none
gelf_short_message_key
Specify the key to use as the short message in gelf format.
none
gelf_timestamp_key
Specify the key to use for timestamp in gelf format.
none
header
Add a HTTP header key/value pair. Multiple headers can be set.
none
header_tag
Specify an optional HTTP header field for the original message tag. In v5.1.3 or later, the header is skipped with a warning if the tag contains a carriage return, line feed, or null character.
none
headers_key
Specify the key to use as the headers of the request (must prefix with $). The key must contain a map, which will have the contents merged on the request headers. This can be used for many purposes, such as specifying the content type of the data contained in body_key. In v5.1.3 or later, any header whose name or value contains a carriage return, line feed, or null character is skipped with a warning.
none
host
IP address or hostname of the target HTTP Server.
127.0.0.1
http.read_idle_timeout
Set maximum allowed time between two consecutive reads. If set to 0s, uses the io_timeout value from the network setup.
0s
http.response_timeout
Set maximum time to wait for a server response.
60s
http_method
Specify the HTTP method. Supported values: POST, PUT, and PATCH. PATCH is supported in v5.1.5 or later.
POST
http_passwd
Basic Auth password. Requires http_user to be set.
none
http_user
Basic Auth username.
none
json_date_format
Specify the format of the date. Supported formats: double, epoch, epoch_ms, iso8601, java_sql_timestamp.
none
json_date_key
Specify the name of the time key in the output record. To disable the time key, set the value to false.
date
log_response_payload
Specify if the response payload should be logged or not.
true
oauth2.audience
Optional OAuth 2.0 audience parameter.
none
oauth2.auth_method
OAuth 2.0 client authentication method. Supported values: basic, post, private_key_jwt.
basic
oauth2.client_id
Required when oauth2.enable is true. OAuth 2.0 client ID.
none
oauth2.client_secret
Required when oauth2.enable is true and oauth2.auth_method is basic or post. OAuth 2.0 client secret.
none
oauth2.connect_timeout
Connect timeout for OAuth 2.0 token requests.
0s
oauth2.enable
Enable OAuth 2.0 client credentials for outgoing requests.
false
oauth2.jwt_aud
Audience for private_key_jwt JSON Web Token (JWT) assertion. Defaults to the value of oauth2.token_url when not set.
none
oauth2.jwt_cert_file
Required when oauth2.enable is true and oauth2.auth_method is private_key_jwt. Path to certificate file used by private_key_jwt. Fluent Bit derives the oauth2.jwt_header thumbprint from this certificate.
none
oauth2.jwt_header
JWT header claim name for the private_key_jwt certificate thumbprint. Supported values are kid (hexadecimal SHA-1), x5t (base64url SHA-1), and x5t#S256 (base64url SHA-256). Fluent Bit doesn't validate this value: an unrecognized name is sent as the claim name with a hexadecimal SHA-1 thumbprint.
kid
oauth2.jwt_key_file
Required when oauth2.enable is true and oauth2.auth_method is private_key_jwt. Path to PEM private key file used by private_key_jwt.
none
oauth2.jwt_ttl_seconds
Lifetime in seconds for private_key_jwt JWT client assertions.
300
oauth2.refresh_skew_seconds
Seconds before expiry at which to refresh the access token.
60
oauth2.resource
Optional OAuth 2.0 resource parameter.
none
oauth2.scope
Optional OAuth 2.0 scope.
none
oauth2.timeout
Timeout for OAuth 2.0 token requests. Defaults to http.response_timeout when unset.
0s
oauth2.token_url
Required when oauth2.enable is true. OAuth 2.0 token endpoint URL. Must use the http or https scheme. Use https in production: with http, Fluent Bit sends the client credentials without TLS.
none
oauth2.user_agent
Optional User-Agent header value to include in OAuth 2.0 token requests. If omitted, no User-Agent header is sent.
none
port
TCP port of the target HTTP Server.
80
proxy
Specify an HTTP Proxy. The expected format of this value is http://HOST:PORT. HTTPS isn't supported. It's recommended to configure the HTTP proxy environment variables instead as they support both HTTP and HTTPS.
none
uri
Specify an optional HTTP URI for the target web server. For example, /somepath.
none
TLS / SSL
The HTTP output plugin supports TLS/SSL. For more details about the properties available and general configuration, see TLS/SSL.
Throttle signals
In v5.2 or later, the HTTP output reports a throttle signal when the server responds with HTTP 429, or with HTTP 503 and a valid Retry-After value. If throttling is enabled for the output, Fluent Bit pauses the output for a cooldown and honors the Retry-After delay. Otherwise, Fluent Bit retries these responses like other failures. A 503 response without a valid Retry-After value is always handled as a normal retry.
When body_key and headers_key are set, Fluent Bit sends each record in a separate request. It stops at the first throttle response, and later sends the whole chunk again, including records that the server already accepted, after the cooldown when throttle is enabled or as a normal retry otherwise. The chunk isn't sent again if retry_limit is no_retries, or, when throttle is off, after the normal retries run out. Fluent Bit then discards it, or moves it to the Dead Letter Queue if that's enabled.
Get started
To insert records into an HTTP server, you can run the plugin from the command line or through the configuration file.
Command line
The HTTP plugin can read the parameters from the command line through the -p argument (property) or setting them directly through the service URI. The URI format is the following:
Using the format specified, you could start Fluent Bit through:
Configuration file
In your main configuration file append the following:
By default, the URI becomes tag of the message, the original tag is ignored. To retain the tag, multiple configuration sections have to be made based and flush to more than one URI.
Another supported approach is the sending the original message tag in a configurable header. It's up to the receiver to do what it wants with that header field. For example, parse it and use it as the tag for example.
To configure this behaviour, add this configuration:
Provided you are using Fluentd as data receiver, you can combine in_http and out_rewrite_tag_filter to make use of this HTTP header.
Fluent Bit overrides the tag, which is from URI path, with a custom header.
Add a header
This example adds a header.
Sumo Logic HTTP collector
The following is a suggested configuration for Sumo Logic using json_lines with iso8601 timestamps. The PrivateKey is specific to a configured HTTP collector.
A sample Sumo Logic query for the CPU input. (Requires json_lines format with iso8601 date format for the timestamp field).
Using PUT method
The following example shows how to configure the HTTP output plugin to use the PUT method instead of the default POST method.
Last updated
Was this helpful?