Stackdriver
Stackdriver output plugin allows to ingest your records into Google Cloud Stackdriver Logging service.
Before to get started with the plugin configuration, make sure to obtain the proper credentials to get access to the service. We strongly recommend to use a common JSON credentials file, reference link:
Your goal is to obtain a credentials JSON file that will be used later by Fluent Bit Stackdriver output plugin.
Configuration Parameters
Configuration File
If you are using a Google Cloud Credentials File, the following configuration is enough to get started:
Example configuration file for k8s resource type:
local_resource_id is used by stackdriver output plugin to set the labels field for different k8s resource types. Stackdriver plugin will try to find the local_resource_id field in the log entry. If there is no field logging.googleapis.com/local_resource_id in the log, the plugin will then construct it by using the tag value of the log.
The local_resource_id should be in format:
k8s_container.<namespace_name>.<pod_name>.<container_name>
k8s_node.<node_name>
k8s_pod.<namespace_name>.<pod_name>
This implies that if there is no local_resource_id in the log entry then the tag of logs should match this format. Note that we have an option tag_prefix so it is not mandatory to use k8s_container(node/pod) as the prefix for tag.
Troubleshooting Notes
Upstream connection error
Github reference: #761
An upstream connection error means Fluent Bit was not able to reach Google services, the error looks like this:
This belongs to a network issue by the environment where Fluent Bit is running, make sure that from the Host, Container or Pod you can reach the following Google end-points:
Fail to process local_resource_id
The error looks like this:
Do following check:
If the log entry does not contain the local_resource_id field, does the tag of the log match for format?
If tag_prefix is configured, does the prefix of tag specified in the input plugin match the tag_prefix?
Other implementations
Stackdriver officially supports a logging agent based on Fluentd.
We plan to support some special fields in structured payloads. Use cases of special fields is here.
Last updated