Logs to metrics
Generate metrics from logs
Supported event types: logs
The log to metrics filter lets you generate log-derived metrics. It supports modes to count records, provide a gauge for field values, or create a histogram. You can also match or exclude specific records based on regular expression patterns for values or nested values.
This filter doesn't actually act as a record filter and therefore doesn't change or drop records. All records will pass through this filter untouched, and any generated metrics will be emitted into a separate metric pipeline.
Configuration parameters
The plugin supports the following configuration parameters:
add_label
Adds a custom label NAME and set the value to the value of KEY.
none
NAME KEY
bucket
Optional for metric_mode histogram. If not set, default Prometheus-style buckets are used.
none
For example, 0.75.
discard_logs
Flag that defines if logs should be discarded after processing. This applies for all logs, whether they have emitted metrics or not.
false
emitter_mem_buf_limit
Buffer limit for the internal metrics emitter. When the buffer is full, the emitter pauses and incoming metric records can be dropped. Monitor per-emitter activity through the /api/v1/metrics endpoint using the configured emitter_name.
10M
emitter_name
Name of the emitter (advanced users).
none
exclude
Excludes records in which the content of KEY matches the regular expression REGEX.
none
KEY REGEX
flush_interval_nsec
The interval for metrics emission, in nanoseconds. This parameter works in conjunction with flush_interval_sec.
0
flush_interval_sec
The interval for metrics emission, in seconds. If flush_interval_sec and flush_interval_nsec are either both unset or both set to 0, the filter emits metrics immediately after each filter match. Otherwise, if either parameter is set to a non-zero value, the filter emits metrics at the specified interval. Longer intervals help lower resource consumption in high-load situations.
0
kubernetes_mode
If enabled, adds pod_id, pod_name, namespace_name, docker_id and container_name to the metric as labels. This option is intended to be used in combination with the Kubernetes filter plugin, which fills those fields.
false
label_field
Includes a record field as label dimension in the metric. Each unique combination of label values creates a separate time series, so high-cardinality fields (such as request paths, hostnames, or user IDs) can grow memory use without bound in long-running pipelines.
none
Name of record key. Supports record accessor notation for nested fields.
metric_description
Sets a description for the metric.
none
metric_mode
Defines the mode for the metric. Valid values are counter, gauge or histogram.
counter
metric_name
Sets the name of the metric.
a
metric_namespace
Sets the namespace of the metric.
log_metric
metric_subsystem
Subsystem of the metric.
none
regex
Includes records in which the content of KEY matches the regular expression REGEX.
none
KEY REGEX
tag
Defines the tag for the generated metrics record.
none
value_field
Required for modes gauge and histogram. Specifies the record field that holds a numerical value. Ignored for counter mode, which increments by 1 per matching record.
none
Name of record key. Supports record accessor notation for nested fields.
Be careful with label cardinality. The counter and gauge modes accumulate one series per unique combination of label values for the lifetime of the process. Labelling on unbounded fields such as request paths, hostnames, or user IDs can fill the emitter buffer and stop metrics from being produced. Prefer pre-aggregating high-cardinality fields upstream, or use histogram mode where bucketing limits growth.
Examples
The following examples assume Prometheus is running on the local machine as shown in the Fluent Bit configurations.
Counter
The following example takes records from two dummy inputs and counts all messages that pass through the log_to_metrics filter. It then generates metric records, which are provided to the prometheus_exporter output:
Run this configuration file with Prometheus to collect the metrics from the Fluent Bit configurations.
You can then use a tool like curl to retrieve the generated metric:
Gauge
The gauge mode needs a value_field to specify where to generate the metric values from. This example also applies a regex filter and enables the kubernetes_mode option:
Run this configuration file with Prometheus to collect the metrics from the Fluent Bit configurations.
You can then use a tool like curl to retrieve the generated metric:
In the resulting output, only one line is printed. Records from the first input plugin are ignored because they don't match the regular expression.
This filter also lets you use multiple rules, which are applied in order. You can have as many regex and exclude entries as required (see Grep filter plugin).
If you execute the example curl command multiple times, the example metric value remains at 60 because the messages generated by the Dummy plugin don't change. In a real-world scenario, the values would change and return to the last processed value.
Metric label_values
label_valuesThe label sets defined by add_label and label_field are added to the metric. The lines in the metric represent every combination of labels. Only combinations that are actually used are displayed here.
Histogram
Similar to the gauge mode, the histogram mode needs a value_field to specify where to generate the metric values from. This example also applies a regex filter and enables the kubernetes_mode option:
Run this configuration file with Prometheus to collect the metrics from the Fluent Bit configurations.
You can then use a tool like curl to retrieve the generated metric:
In the resulting output, there are several buckets by default: 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0, 10.0 and +Inf. Values are sorted into these buckets. A sum and a counter are also part of this metric. You can specify own buckets in the configuration, like in the following example:
Run this configuration file with Prometheus to collect the metrics from the Fluent Bit configurations.
The +Inf bucket will always be included regardless of the buckets you specify. The buckets in a histogram are cumulative, so a value added to one bucket will be added to all larger buckets, too.
This filter also attaches Kubernetes labels to each metric, identical to the behavior of label_field. This results in two sets for the histogram.
Operational considerations
This filter holds metric state in memory for the lifetime of the Fluent Bit process. Long-running pipelines can run into issues that don't appear in short-lived tests.
Cardinality and process-lifetime state
The counter and gauge modes maintain one time series per unique combination of label values. State is held for the lifetime of the process, with no eviction or expiry.
Histogram mode keeps the same per-combination state, but bucket counts within a series are fixed by the bucket configuration. Memory growth per new label combination is bounded.
Plan label budgets before deploying. Avoid using high-cardinality fields such as request paths, full URLs, user IDs, or source IP addresses as direct labels. Pre-aggregate or normalize these fields upstream with a Lua or modify filter before they reach log_to_metrics.
Monitoring emitter health
Each log_to_metrics filter instance creates an internal emitter input. The instance name comes from emitter_name, or is auto-generated as emitter_for_<filter> when omitted.
Enable the HTTP monitoring server in your service configuration and inspect emitter activity:
Look for the input section that matches your emitter name. The records and bytes counters increasing over time confirms the filter is producing metrics. Counters that flatten to zero while the source input keeps growing indicate the emitter has stalled, most often due to buffer pressure.
Buffer pressure and back pressure
When emitter_mem_buf_limit is reached, the emitter pauses ingestion using standard mem_buf_limit semantics. The source filter continues to run, but new metric updates can't flow downstream until the buffer drains.
To mitigate buffer pressure:
Increase
emitter_mem_buf_limitif memory budget allows.Reduce label cardinality by pre-aggregating upstream.
Set a non-zero
flush_interval_secto batch emissions and reduce per-record overhead.
Counter resets on restart
Counters reset to zero whenever Fluent Bit restarts. State isn't persisted between runs.
This is normal Prometheus counter behavior. Consumers should query counters with rate functions such as rate() or increase() rather than reading absolute values, so restarts appear as expected discontinuities rather than data loss.
Last updated
Was this helpful?