> For the complete documentation index, see [llms.txt](https://docs.fluentbit.io/manual/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.fluentbit.io/manual/data-pipeline/inputs/opentelemetry.md). # OpenTelemetry An input plugin to ingest OpenTelemetry logs, metrics, and traces {% hint style="info" %} **Supported event types:** `logs` `metrics` `traces` {% endhint %} The OpenTelemetry input plugin lets you receive data based on the OpenTelemetry specification from various OpenTelemetry exporters, the OpenTelemetry Collector, or the Fluent Bit OpenTelemetry output plugin. Fluent Bit has a compliant implementation which fully supports `OTLP/HTTP` and `OTLP/GRPC`. The single `port` configured defaults to `4318` and supports both transport methods. ## Configuration The following table includes both: * settings specific to the OpenTelemetry input plugin * shared `http_server.*` listener settings that are used by several HTTP-based inputs For a cross-plugin explanation of the shared listener settings, see [Shared HTTP listener settings for inputs](/manual/administration/configuring-fluent-bit/yaml/pipeline-section.md#shared-http-listener-settings-for-inputs). | Key | Description | Default | | --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- | | `alias` | Sets an alias for multiple instances of the same input plugin. If no alias is specified, a default name will be assigned using the plugin name followed by a dot and a sequence number. | *none* | | `buffer_chunk_size` | Size of each buffer chunk allocated for HTTP requests (advanced users only). | `512K` | | `buffer_max_size` | Maximum size of an OTLP request payload, as received and after decompression. Compatibility alias for `http_server.buffer_max_size`. See [Request size limits](#request-size-limits). | 64 MiB | | `encode_profiles_as_log` | Encode profiles received as text and ingest them in the logging pipeline. | `true` | | `host` | The hostname. | `localhost` | | `http2` | Enable HTTP/2 protocol support for the OpenTelemetry receiver. | `true` | | `http_server.max_connections` | Maximum number of concurrent active HTTP connections. `0` means unlimited. | `0` | | `http_server.workers` | Number of HTTP listener worker threads. | `1` | | `http_server.ingress_queue_event_limit` | Maximum number of deferred ingress queue entries. Applies only when `http_server.workers` is greater than `1`. | `8192` | | `http_server.ingress_queue_byte_limit` | Maximum size of the deferred ingress queue. Applies only when `http_server.workers` is greater than `1`. | `256M` | | `http_server.max_decoded_body_size` | Maximum size of an `OTLP/HTTP` body after decompression. Defaults to `buffer_max_size`. Supported in v5.2 or later. See [Request size limits](#request-size-limits). | 64 MiB | | `listen` | The network address to listen on. | `0.0.0.0` | | `log_level` | Specifies the log level for this plugin. If not set here, the plugin uses the global log level specified in the `service` section of your configuration file. | `info` | | `log_suppress_interval` | Suppresses log messages from this plugin that appear similar within a specified time interval. `0` no suppression. | `0` | | `logs_body_key` | Specify a body key. | *none* | | `logs_max_size` | Maximum size of the MessagePack data generated for the logs in a single request, including group metadata. Requests that exceed this limit are rejected. Supported in v5.1.5 or later. | `100M` | | `logs_metadata_key` | Key name to store OpenTelemetry logs metadata in the record. | `otlp` | | `mem_buf_limit` | Set a memory buffer limit for the input plugin. If the limit is reached, the plugin will pause until the buffer is drained. The value is in bytes. If set to 0, the buffer limit is disabled. | `0` | | `net.accept_timeout` | Set maximum time allowed to establish an incoming connection. This time includes the TLS handshake. | `10s` | | `net.accept_timeout_log_error` | On client accept timeout, specify if it should log an error. When disabled, the timeout is logged as a debug message. | `true` | | `net.backlog` | Set the backlog size for listening sockets. | `128` | | `net.io_timeout` | Set maximum time a connection can stay idle. | `0s` | | `net.keepalive` | Enable or disable keepalive support. | `true` | | `net.share_port` | Allow multiple plugins to bind to the same port. | `false` | | `oauth2.allowed_audience` | Audience claim to enforce when validating incoming `OAuth 2.0` JSON Web Token (`JWT`) tokens. | *none* | | `oauth2.allowed_clients` | Authorized `client_id` or `azp` claim values. Can be specified multiple times. | *none* | | `oauth2.issuer` | Expected issuer (`iss`) claim for `OAuth 2.0` `JWT` validation. | *none* | | `oauth2.jwks_buffer_max_size` | Maximum `JWKS` response size, headers and body included, `4096` bytes to `16M`. Supported in v5.2 or later. | `1M` | | `oauth2.jwks_refresh_interval` | Seconds before the next validating request refetches `JWKS` keys. No background fetch. See [limits](/manual/administration/configuring-fluent-bit/yaml/pipeline-section.md#jwks-fetch-limits). | `300` | | `oauth2.jwks_response_timeout` | `JWKS` response time limit after the request is sent, `1s` to `300s`. v5.2 or later. See [limits](/manual/administration/configuring-fluent-bit/yaml/pipeline-section.md#jwks-fetch-limits). | `30s` | | `oauth2.jwks_url` | `JWKS` endpoint URL used to fetch public keys for `OAuth 2.0` `JWT` validation. | *none* | | `oauth2.validate` | Enable `OAuth 2.0` `JWT` validation for incoming requests. | `false` | | `port` | The port for Fluent Bit to listen for incoming connections. | `4318` | | `profiles_support` | This is an experimental feature, feel free to test it but don't enable this in production environments. | `false` | | `protobuf_decode_max_size` | Maximum estimated memory to decode the Protocol Buffers payloads in one request. Must be a positive size. Supported in v5.2 or later. See [Request size limits](#request-size-limits). | 128 MiB | | `raw_traces` | Forward JSON traces without processing. When set to `false` (default), JSON traces are processed using the unified JSON parser with strict validation. When set to `true`, JSON trace data is forwarded as raw log messages without validation or processing. Protocol Buffers and gRPC traces are always decoded as traces. | `false` | | `routable` | If set to `true`, the data generated by the plugin will be routable, meaning that it can be forwarded to other plugins or outputs. If set to `false`, the data will be discarded. | `true` | | `storage.pause_on_chunks_overlimit` | Enable pausing on an input when they reach their chunks limit. | *none* | | `storage.type` | Sets the storage type for this input, one of: `filesystem`, `memory` or `memrb`. | `memory` | | `successful_response_code` | Allows for setting a successful response code. Supported values: `200`, `201`, or `204`. | `201` | | `tag` | Set a tag for the events generated by this input plugin. | *none* | | `tag_from_uri` | If no explicit `tag` is set, the tag is created from the URI. For example, `v1_metrics` from `/v1/metrics`. When an explicit `tag` is configured, it takes precedence over the URI-derived tag regardless of this setting. | `true` | | `tag_key` | Record accessor key to use for generating tags from incoming records. | *none* | | `thread.ring_buffer.capacity` | Number of slots in the ring buffer for data entries when running in [threaded](/manual/administration/multithreading.md) mode. Each slot can hold one data entry. | `1024` | | `thread.ring_buffer.window` | Percentage threshold (1-100) of the ring buffer capacity at which a flush is triggered when running in [threaded](/manual/administration/multithreading.md) mode. | `5` | | `threaded` | Enable [multithreading](/manual/administration/multithreading.md) for this input to run in a separate dedicated thread. | `false` | | `tls` | Enable or disable TLS/SSL support. | `off` | | `tls.ca_file` | Absolute path to CA certificate file. | *none* | | `tls.ca_path` | Absolute path to scan for certificate files. | *none* | | `tls.ciphers` | Specify TLS ciphers up to TLSv1.2. | *none* | | `tls.crl_file` | Absolute path to a Certificate Revocation List (CRL) file in PEM format. When set, revoked certificates are rejected during TLS verification. Supported in v5.1.3 or later. | *none* | | `tls.crt_file` | Absolute path to Certificate file. | *none* | | `tls.debug` | Set TLS debug level. Accepts `0` (No debug), `1`(Error), `2` (State change), `3` (Informational) and `4` (Verbose). | `1` | | `tls.key_file` | Absolute path to private Key file. | *none* | | `tls.key_passwd` | Optional password for tls.key\_file file. | *none* | | `tls.max_version` | Specify the maximum version of TLS. | *none* | | `tls.min_version` | Specify the minimum version of TLS. | *none* | | `tls.verify` | Force certificate validation. | `on` | | `tls.verify_hostname` | Enable or disable to verify hostname. | `off` | | `tls.vhost` | Hostname to be used for TLS SNI extension. | *none* | The `http_server.ingress_queue_event_limit` and `http_server.ingress_queue_byte_limit` settings matter only when `http_server.workers` is greater than `1`. When `raw_traces` is set to `false` (default), the traces endpoint (`/v1/traces`) processes incoming trace data using the unified JSON parser with strict validation. The endpoint accepts both `protobuf` and `JSON` encoded payloads. When `raw_traces` is set to `true`, JSON payloads sent to the traces endpoint are packed and forwarded as a log message without processing, validation, or conversion to the Fluent Bit internal trace format. `raw_traces` doesn't apply to Protocol Buffers or gRPC payloads: they still go through the checks in [Request size limits](#request-size-limits), including the decode budget, and are decoded as traces. ### HTTP workers When `http_server.workers` is greater than `1`, each worker decodes incoming requests and places the data in a deferred ingress queue, which the input then consumes. #### Log processing in HTTP workers If every processor in the input's `logs` processor chain can run in a worker, the chain runs inside the HTTP worker that received the request, and the processed logs are added to the ingress queue. The following processors can run in HTTP workers: * [Content modifier](/manual/data-pipeline/processors/content-modifier.md) * [SQL](/manual/data-pipeline/processors/sql.md) If the chain includes any other processor, or a filter used as a processor, the whole chain runs after the logs leave the ingress queue, the same as with a single worker. You don't need to change any configuration to use either mode. #### Full ingress queue The deferred ingress queue is limited by `http_server.ingress_queue_event_limit` and an effective byte limit. The effective byte limit is `http_server.ingress_queue_byte_limit`, or `mem_buf_limit` when `mem_buf_limit` is set to a lower value. The plugin rejects a request in either of these cases: * **The queue is full**: The queue has reached one of its limits and doesn't free enough room within a short wait. * **An event is too large**: A single event that the plugin adds to the queue exceeds the effective byte limit. Each event holds the data from one OTLP message, so one request can add several events to the queue. In both cases, the plugin returns the following responses: * **HTTP/1.1 and HTTP/2**: Returns `503 Service Unavailable` with the message `server overloaded: deferred ingress queue is full`. * **gRPC**: Returns gRPC status `14 (UNAVAILABLE)` with the message `server overloaded`. Clients can't tell the two cases apart from the response. Retry rejected requests with backoff, because a full queue is temporary. If a request keeps failing after several retries, an event it produces probably exceeds the effective byte limit, and retrying won't succeed. Send smaller requests, or raise the effective byte limit by increasing `http_server.ingress_queue_byte_limit` and, if it's set, `mem_buf_limit`. ### OpenTelemetry transport protocol endpoints Fluent Bit exposes the following endpoints for data ingestion based on the OpenTelemetry protocol: For `OTLP/HTTP`: * Logs * `/v1/logs` * Metrics * `/v1/metrics` * Traces * `/v1/traces` For `OTLP/GRPC`: * Logs * `/opentelemetry.proto.collector.log.v1.LogService/Export` * `/opentelemetry.proto.collector.log.v1.LogService/Export` * Metrics * `/opentelemetry.proto.collector.metric.v1.MetricService/Export` * `/opentelemetry.proto.collector.metrics.v1.MetricsService/Export` * Traces * `/opentelemetry.proto.collector.trace.v1.TraceService/Export` * `/opentelemetry.proto.collector.traces.v1.TracesService/Export` ### Request size limits In v5.2 or later, the plugin applies the following limits to each request: * `buffer_max_size` limits the request payload. It defaults to 64 MiB (67,108,864 bytes), the size recommended by the OTLP specification. Earlier versions used the shared `4M` default. Fluent Bit adds 64 KiB to the HTTP request buffer for headers and gRPC framing. An `OTLP/HTTP` request whose payload is larger than `buffer_max_size`, before or after decompression, is rejected with `413`. When an uncompressed payload reaches the plugin, the response includes the message `OTLP request size exceeds limit`. A request larger than the HTTP request buffer (`buffer_max_size` plus 64 KiB) is rejected by the listener before it reaches the plugin. On HTTP/1.1, the listener sends a `413` response with an empty body and closes the connection. On HTTP/2, the request fails without a `413` response. A compressed body that exceeds the limit during decompression gets a `413` response with an empty body. * `http_server.max_decoded_body_size` limits an `OTLP/HTTP` body while it's decompressed. If it isn't set, it equals the `buffer_max_size` value. Setting it to `0` selects `100M`. Because the decompressed payload is also checked against `buffer_max_size`, a higher value doesn't allow larger payloads. * `protobuf_decode_max_size` limits the estimated memory needed to decode the Protocol Buffers payload of a request, summed across all of its gRPC messages. The default is 128 MiB (134,217,728 bytes). Fluent Bit checks this budget before it ingests any data from the request. A request over the budget is rejected with `413` or gRPC status `8 (RESOURCE_EXHAUSTED)`, and a malformed payload with `400` or gRPC status `3 (INVALID_ARGUMENT)`. Both use the message `protobuf decode budget exceeded or invalid payload`. The same value also limits the memory used to convert each Protocol Buffers logs message of 8 KiB or more. A logs message that exceeds it during conversion fails with `400` or a gRPC error. JSON payloads aren't affected. Size values such as `64M` use decimal units. To set exactly 64 MiB, use `67108864`. If you raise `buffer_max_size`, you might also need to raise `protobuf_decode_max_size`, because larger payloads need more memory to decode. ### gRPC message size limits In v5.1.4 or later, the plugin enforces the following limits on `OTLP/GRPC` requests: * Each gRPC message can't exceed `buffer_max_size` in v5.2 or later, or 16 MiB in earlier versions, both as received and after decompression. * The combined size of all messages in a request, after decompression, can't exceed `buffer_max_size`. Messages compressed with `gzip`, `zstd`, or `snappy` (set by the `grpc-encoding` header) are decompressed in bounded steps, so a highly compressed payload is rejected as soon as its expanded size reaches the limit. Requests that exceed a limit are rejected with gRPC status `8 (RESOURCE_EXHAUSTED)` and the message `gRPC message size exceeds limit` or `expanded gRPC payload size exceeds limit`. Compressed messages that use any other `grpc-encoding` value are rejected with gRPC status `12 (UNIMPLEMENTED)`. Uncompressed messages aren't affected by the `grpc-encoding` value. ## Get started The OpenTelemetry input plugin supports the following telemetry data types: | Type | HTTP1/JSON | HTTP1/Protobuf | HTTP2/gRPC | | ------- | ---------- | -------------- | ---------- | | Logs | Stable | Stable | Stable | | Metrics | Stable | Stable | Stable | | Traces | Stable | Stable | Stable | A sample configuration file to get started will look something like the following: {% tabs %} {% tab title="fluent-bit.yaml" %} ```yaml pipeline: inputs: - name: opentelemetry listen: 127.0.0.1 port: 4318 outputs: - name: stdout match: '*' ``` {% endtab %} {% tab title="fluent-bit.conf" %} ``` [INPUT] Name opentelemetry Listen 127.0.0.1 Port 4318 [OUTPUT] Name stdout Match * ``` {% endtab %} {% endtabs %} With this configuration, Fluent Bit listens on port `4318` for data. You can now send telemetry data to the endpoints `/v1/metrics` for metrics, `/v1/traces` for traces, and `/v1/logs` for logs. A sample curl request to POST JSON encoded log data would be: ```shell curl --header "Content-Type: application/json" --request POST --data '{"resourceLogs":[{"resource":{},"scopeLogs":[{"scope":{},"logRecords":[{"timeUnixNano":"1660296023390371588","body":{"stringValue":"{\"message\":\"dummy\"}"},"traceId":"","spanId":""}]}]}]}' http://0.0.0.0:4318/v1/logs ``` ## OpenTelemetry trace improvements Fluent Bit includes enhanced support for OpenTelemetry traces with improved JSON parsing, error handling, and validation capabilities. ### Unified trace JSON parser Fluent Bit provides a unified interface for processing OpenTelemetry trace data in JSON format. The parser converts OpenTelemetry JSON trace payloads into the Fluent Bit internal trace representation, supporting the full OpenTelemetry trace specification including: * Resource spans with attributes * Instrumentation scope information * Span data (names, IDs, timestamps, status) * Span events and links * Trace and span ID validation The unified parser handles the OpenTelemetry JSON encoding format, which wraps attribute values in type-specific containers (for example, `stringValue`, `intValue`, `doubleValue`, `boolValue`). ### Error status propagation The OpenTelemetry input plugin provides detailed error status information when processing trace data. If trace processing fails, the plugin returns specific error codes that help identify the issue: * `FLB_OTEL_TRACES_ERR_INVALID_JSON` - Invalid JSON format * `FLB_OTEL_TRACES_ERR_INVALID_TRACE_ID` - Invalid trace ID format or length * `FLB_OTEL_TRACES_ERR_INVALID_SPAN_ID` - Invalid span ID format or length * `FLB_OTEL_TRACES_ERR_INVALID_PARENT_SPAN_ID` - Invalid parent span ID * `FLB_OTEL_TRACES_ERR_STATUS_FAILURE` - Invalid span status code * `FLB_OTEL_TRACES_ERR_INVALID_ATTRIBUTES` - Invalid attribute format * `FLB_OTEL_TRACES_ERR_INVALID_EVENT_ENTRY` - Invalid span event * `FLB_OTEL_TRACES_ERR_INVALID_LINK_ENTRY` - Invalid span link #### Valid span status codes The OpenTelemetry specification defines three valid span status codes. When processing trace data, the plugin accepts the following status code values (case-insensitive): * `OK` - The operation completed successfully * `ERROR` - The operation has an error * `UNSET` - The status isn't set (default) Any other status code value triggers `FLB_OTEL_TRACES_ERR_STATUS_FAILURE` and causes the trace data to be rejected. The status code must be provided as a string in the `status.code` field of the span object. #### Error handling behavior When trace validation fails, the following behavior applies: * **Trace data is dropped**: Invalid trace data isn't processed or forwarded. The trace payload is rejected immediately. * **Error logging**: The plugin logs an error message with the specific error status code to help diagnose issues. Error messages include the error code number and description. * **No retry mechanism**: Failed requests aren't automatically retried. The client must resend corrected trace data. * **HTTP response codes**: * **HTTP/1.1**: Returns `400 Bad Request` with an error message when validation fails. Returns the configured `successful_response_code` (default `201 Created`) when processing succeeds. * **gRPC**: Returns gRPC status `2 (UNKNOWN)` with message "Serialization error." when validation fails. Returns gRPC status `0 (OK)` with an empty `ExportTraceServiceResponse` when processing succeeds. ### Strict ID decoding Fluent Bit enforces strict validation for trace and span IDs to ensure data integrity: * **Trace IDs**: Must be exactly 32 hexadecimal characters (16 bytes) * **Span IDs**: Must be exactly 16 hexadecimal characters (8 bytes) * **Parent Span IDs**: Must be exactly 16 hexadecimal characters (8 bytes) when present The validation process: 1. Verifies the ID length matches the expected size 2. Validates that all characters are valid hexadecimal digits (0-9, a-f, A-F) 3. Decodes the hexadecimal string to binary format 4. Rejects invalid IDs with appropriate error codes Invalid IDs result in error status codes (`FLB_OTEL_TRACES_ERR_INVALID_TRACE_ID`, `FLB_OTEL_TRACES_ERR_INVALID_SPAN_ID`, and so on) and the trace data is rejected to prevent processing of corrupted or malformed trace information. ### Example: JSON trace payload The following example shows a valid OpenTelemetry JSON trace payload that can be sent to the `/v1/traces` endpoint: ```json { "resourceSpans": [ { "resource": { "attributes": [ { "key": "service.name", "value": { "stringValue": "my-service" } } ] }, "scopeSpans": [ { "scope": { "name": "my-instrumentation", "version": "1.0.0" }, "spans": [ { "traceId": "0123456789abcdef0123456789abcdef", "spanId": "0123456789abcdef", "name": "my-span", "kind": 1, "startTimeUnixNano": "1660296023390371588", "endTimeUnixNano": "1660296023391371588", "status": { "code": "OK" }, "attributes": [ { "key": "http.method", "value": { "stringValue": "GET" } } ] } ] } ] } ] } ``` Trace IDs must be exactly 32 hex characters and span IDs must be exactly 16 hex characters. Invalid IDs will be rejected with appropriate error messages. In the example, the `status.code` field uses `"OK"`. Valid status code values are `"OK"`, `"ERROR"`, and `"UNSET"` (case-insensitive). Any other value triggers `FLB_OTEL_TRACES_ERR_STATUS_FAILURE` and causes the trace to be rejected. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.fluentbit.io/manual/data-pipeline/inputs/opentelemetry.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.