> For the complete documentation index, see [llms.txt](https://docs.fluentbit.io/manual/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.fluentbit.io/manual/data-pipeline/inputs/prometheus-textfile.md). # Prometheus text file {% hint style="info" %} **Supported event types:** `metrics` {% endhint %} The *Prometheus text file* input plugin allows Fluent Bit to read metrics from Prometheus text format files (`.prom` files) on the local filesystem. Use this plugin to collect custom metrics that are written to files by external applications or scripts, similar to the Prometheus Node Exporter text file collector. ## Configuration parameters | Key | Description | Default | | ----------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | | `alias` | Sets an alias. Use for multiple instances of the same input plugin. If no alias is specified, a default name is assigned using the plugin name followed by a dot and a sequence number. | *none* | | `log_level` | Specifies the log level for input plugin. If not set here, plugin uses global log level in `service` section. | `info` | | `log_suppress_interval` | Suppresses log messages from input plugin that appear similar within a specified time interval. `0` no suppression. | `0` | | `mem_buf_limit` | Set a memory buffer limit for the input plugin. If the limit is reached, the plugin will pause until the buffer is drained. The value is in bytes. If set to 0, the buffer limit is disabled. | `0` | | `path` | Comma-separated list of files or glob patterns to read. Supports `*` wildcard (for example, `/var/lib/prometheus/*.prom`). | *none* | | `routable` | If set to `true`, the data generated by the plugin will be routable, meaning that it can be forwarded to other plugins or outputs. If set to `false`, the data will be discarded. | `true` | | `scrape_interval` | Interval between file scans. | `10s` | | `storage.pause_on_chunks_overlimit` | Enable pausing on an input when they reach their chunks limit. | *none* | | `storage.type` | Sets the storage type for this input, one of: `filesystem`, `memory` or `memrb`. | `memory` | | `tag` | Set a tag for the events generated by this input plugin. | *none* | | `thread.ring_buffer.capacity` | Set custom ring buffer capacity when the input runs in threaded mode. | `1024` | | `thread.ring_buffer.window` | Set custom ring buffer window percentage for threaded inputs. | `5` | | `threaded` | Enable threading on an input. | `false` | ## Get started ### Basic configuration The following configuration reads the files that match `/var/lib/prometheus/textfile/*.prom` every 15 seconds: ```yaml pipeline: inputs: - name: prometheus_textfile tag: custom_metrics path: '/var/lib/prometheus/textfile/*.prom' scrape_interval: 15s outputs: - name: prometheus_exporter match: custom_metrics host: 192.168.100.61 port: 2021 ``` ## Prometheus text format The plugin expects files to be in the standard Prometheus text exposition format. Here's an example of a valid `.prom` file: ``` # HELP custom_counter_total A custom counter metric # TYPE custom_counter_total counter custom_counter_total{instance="server1",job="myapp"} 42 # HELP custom_gauge A custom gauge metric # TYPE custom_gauge gauge custom_gauge{environment="production"} 1.23 # HELP custom_histogram_bucket A custom histogram # TYPE custom_histogram_bucket histogram custom_histogram_bucket{le="0.1"} 10 custom_histogram_bucket{le="0.5"} 25 custom_histogram_bucket{le="1.0"} 40 custom_histogram_bucket{le="+Inf"} 50 custom_histogram_sum 125.5 custom_histogram_count 50 ``` ## Scrape behavior Every `scrape_interval`, the plugin reads each file that matches `path`. Only regular files are read. Empty files are skipped. On Linux and macOS, directories that match `path` are skipped without an error, so set `path` to a file or a glob pattern such as `/var/lib/prometheus/textfile/*.prom`, not to a directory. On Windows, each `path` entry must contain a `*` wildcard; otherwise the plugin logs `path has no wild card` and the scrape reports an error. In v5.2 or later, a pattern that matches no files is skipped and logged only at the `debug` level. On Linux and macOS, this also applies when the directory in the pattern is missing or can't be read, so a typo in `path` doesn't set the error gauge. Earlier versions logged a warning and an error. ### Merged scrape payload In v5.2 or later, the plugin merges the metrics from all files into one metrics payload for each scrape. Earlier versions sent a separate payload for each file. Merging changes how metrics with the same name in more than one file are handled: * Metrics with the same name, type, and `HELP` text are combined into one metric family. * If the same counter, gauge, untyped, or summary series appears in more than one file, only one value is kept. Files are read in no particular order, so which value is kept isn't defined. * If the same histogram series appears in more than one file, its buckets, count, and sum are added together. * If a later file defines a combined metric with more labels, or with a different number of differently named labels, the merge fails. If it has the same number of labels with different names, or fewer labels with the same names, its values are merged silently under the first file's label names. * If a later file defines a combined histogram with a different number of buckets, the merge fails. With the same number of buckets but different bucket boundaries, the bucket values are added together silently. * Metrics with the same name but a different type or `HELP` text aren't combined and are kept as separate families with the same name. Some outputs, such as [Prometheus exporter](/manual/data-pipeline/outputs/prometheus-exporter.md), reject these as conflicting. If a merge fails, the plugin logs `could not merge metrics from ` and discards the whole scrape, including the error gauge. The same files fail on every scrape until they change. Define each metric in only one file to avoid these cases. ### Scrape error gauge In v5.2 or later, each scrape also includes the following gauge: | Metric | Labels | Description | | --------------------------------- | ------- | ------------------------------------------------------------------------------ | | `fluentbit_textfile_scrape_error` | `input` | `1` if an error occurred while reading files during the scrape, otherwise `0`. | The gauge is set to `1` when a glob pattern can't be read, a matched file can't be checked or read, or a file isn't valid Prometheus text format. Metrics from files that fail are left out of the scrape, and metrics from the other files are still sent. A file that can't be read is logged only at the `debug` level, so check the gauge rather than the logs. The gauge is sent even when no file contributes metrics, except when the scrape is discarded. Alert when the gauge is `1`, when it's missing, and when the log contains `could not merge metrics`. Some outputs keep showing the last value they received. [Prometheus exporter](/manual/data-pipeline/outputs/prometheus-exporter.md) keeps serving each input's previous scrape until Fluent Bit restarts or reloads its configuration, so after a merge failure the gauge can stay at `0` instead of disappearing. The `input` label is the internal name of the input instance, such as `prometheus_textfile.0`, even if you set an `alias`, so `alias` doesn't change this label. Because it's a new metric in v5.2, outputs and dashboards that only allow known metric names might need to allow it. ### File access The plugin follows symbolic links and reads any matching file that the Fluent Bit user can read, including files outside the directory in `path`. Use a specific pattern, and keep the files in a directory that only trusted processes can write to. Write each file to a temporary name that doesn't match `path`, and then rename it. A scrape that reads a partly written file can report a parse error for that file. It can also read content that's valid but incomplete, so some series are missing or a value is cut short, without any error. Make the file readable by the user that runs Fluent Bit before you rename it, because tools such as `mktemp` create files that only their owner can read. The following examples use `chmod 0644`, which lets every local user read the file. To limit access, set the file's group to one that the Fluent Bit user belongs to, and use `0640` instead. ## Use cases ### Custom application metrics Applications can write custom metrics to `.prom` files, and this plugin will collect them: ```bash # Script writes metrics to a unique temporary file, then renames it TMP=$(mktemp /var/lib/prometheus/textfile/app.prom.XXXXXX) echo "# HELP app_requests_total Total HTTP requests" > "$TMP" echo "# TYPE app_requests_total counter" >> "$TMP" echo "app_requests_total{status=\"200\"} 1500" >> "$TMP" echo "app_requests_total{status=\"404\"} 23" >> "$TMP" # Let the Fluent Bit user read the file before publishing it chmod 0644 "$TMP" mv "$TMP" /var/lib/prometheus/textfile/app.prom ``` ### Batch job metrics Cron jobs or batch processes can write completion metrics: ```bash #!/bin/bash # Backup script writes completion metrics BACKUP_START=$(date +%s) # ... perform backup ... BACKUP_END=$(date +%s) DURATION=$((BACKUP_END - BACKUP_START)) TMP=$(mktemp /var/lib/prometheus/textfile/backup.prom.XXXXXX) cat > "$TMP" << EOF # HELP backup_duration_seconds Time taken to complete backup # TYPE backup_duration_seconds gauge backup_duration_seconds ${DURATION} # HELP backup_last_success_timestamp_seconds Last successful backup timestamp # TYPE backup_last_success_timestamp_seconds gauge backup_last_success_timestamp_seconds ${BACKUP_END} EOF # Let the Fluent Bit user read the file before publishing it chmod 0644 "$TMP" mv "$TMP" /var/lib/prometheus/textfile/backup.prom ``` ### System integration External monitoring tools can write metrics that Fluent Bit will collect and forward. ## Integration with other plugins ### OpenTelemetry destination ```yaml pipeline: inputs: - name: prometheus_textfile tag: textfile_metrics path: /var/lib/prometheus/textfile/*.prom - name: node_exporter_metrics tag: system_metrics scrape_interval: 15s outputs: - name: opentelemetry match: '*' host: 192.168.56.4 port: 2021 ``` --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.fluentbit.io/manual/data-pipeline/inputs/prometheus-textfile.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.